The DOI matches its Crossref record; 170 of the 176 references it cites were found in a registry or an index; none of the 220 sources its claims cite is checked for support yet.
A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence
Open accessAbstract
A number of algorithms in the field of artificial intelligence offer poorly interpretable decisions. To disclose the reasoning behind such algorithms, their output can be explained by means of so-called evidence-based (or factual) explanations. Alternatively, contrastive and counterfactual explanations justify why the output of the algorithms is not any different and how it could be changed, respectively. It is of crucial importance to bridge the gap between theoretical approaches to contrastive and counterfactual explanation and the corresponding computational frameworks. In this work we conduct a systematic literature review which provides readers with a thorough and reproducible analysis of the interdisciplinary research field under study. We first examine theoretical foundations of contrastive and counterfactual accounts of explanation. Then, we report the state-of-the-art computational frameworks for contrastive and counterfactual explanation generation. In addition, we analyze how grounded such frameworks are on the insights from the inspected theoretical approaches. As a result, we highlight a variety of properties of the approaches under study and reveal a number of shortcomings thereof. Moreover, we define a taxonomy regarding both theoretical and practical approaches to contrastive and counterfactual explanation.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This systematic literature review maps theoretical accounts and computational frameworks for contrastive and counterfactual explanation in XAI, proposing the umbrella term “contfactual explanation.” It reports a predominantly causal theoretical literature, many task-specific generation frameworks, and a weak link between theory and implementation, with evaluation methods lacking standardization.
Key findings
- From 113 unique primary studies, 74/113 (65.49%) addressed definitions/theoretical accounts (RQ1), 52/113 (46.02%) described generation frameworks (RQ2), and only 13/113 (11.50%) addressed both (RQ3).
- Among theoretical accounts, 49/74 (66.22%) concerned causal explanation; the proposed taxonomy distinguishes causal, non-causal, and hybrid contfactual explanation.
- Among 52 generation frameworks, 31 (59.62%) generated counterfactual explanations, 17 (32.69%) contrastive, and 4 (7.69%) fused both; 42/52 (80.77%) targeted classification. Most were model-agnostic counterfactual methods, and most output numerical/feature-based explanations.
- Evaluation was non-standardized: 17/52 (32.69%) provided no evaluation details; most used objective metrics, few used hybrid automatic+human evaluation. Only 13/113 studies were theoretically grounded, and only 3/13 (23.08%) of those interpreted theory to propose their own definition.
Methods
- Systematic literature review following Kitchenham and Charters, Kitchenham et al., and Wohlin, with planning, conducting, and reporting phases; three RQs on definitions, generation methods, and theory-practice grounding.
- Searched Scopus and Web of Science on October 2, 2020 using six queries (q1–q6) over title, abstract, and author keywords with word stems and NEAR/3/W proximity operators; applied inclusion/exclusion criteria, duplicate removal, and snowballing.
- From 338 candidate primary studies, 136 duplicates removed, 75 filtered as irrelevant, 33 discarded for inclusion criteria, and 19 added via bibliography inspection, yielding 113 unique primary studies; data extraction covered definitions/theories and framework problem, method, output form, and evaluation, plus bibliometric analysis with Mendeley and VOSViewer.
Limitations
- The paper states that no uniform set of evaluation methods has been adopted, that many frameworks lack standardization, and that a gap remains between philosophical accounts and ML-related concepts, making theoretical frameworks poorly applicable to XAI.
- The paper states that definitions are highly task- and domain-specific, that most computational frameworks are non-causal and not rooted in theoretical accounts, and that only 13/113 studies were theoretically grounded.
- Evident from the text: the search was limited to Scopus and WoS and to a single date (October 2, 2020), so 2020 coverage is incomplete; the review also notes that many frameworks are limited to specific models, tasks, or small feature sets, and that user-centered evaluation is rare.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
179 Not checked yet · 55 No basis given · 34 Own result · 18 Own method · 15 Own interpretation
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, a major factor discerning theoretical approaches to contfactual explanation is found to consist in their relation to causation.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. CONTFACTUAL EXPLANATION AS DEFINED IN RELATED THEORIES (ANSWER TO RQ 1 )
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, Fig. 5 shows a rapid rise in the number of publications in the past three years.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > A. BIBLIOMETRIC ANALYSIS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Publication counts and author keyword maps are valid indicators of research attention and interdisciplinarity in the field.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“causal counterfactual making up a majority of the considered theoretical frameworks (see Fig. 8).”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. CONTFACTUAL EXPLANATION AS DEFINED IN RELATED THEORIES (ANSWER TO RQ 1 )
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Counterfactual explanation encompasses contrastive, counterfactual, and contrastive-counterfactual explanation.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Each type of contfactual explanation is present in the findings”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. CONTFACTUAL EXPLANATION AS DEFINED IN RELATED THEORIES (ANSWER TO RQ 1 )
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Counterfactual explanation encompasses contrastive, counterfactual, and contrastive-counterfactual explanation.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As depicted in Fig. 9, most contfactual explanation generation frameworks deal with counterfactual explanation (31 out of 52 frameworks; 59.62%).”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 1) AI PROBLEM
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Note that 13 out of 113 (11.50%) primary studies were found to address both RQ 1 and RQ 2 and therefore answer RQ 3 .”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > E. DATA EXTRACTION AND SYNTHESIS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“One of these studies [129] deals with both classification and regression.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 1) AI PROBLEM
What the sources say
- Not checked yetcites [129] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Only four studies (7.69%) fuse contrastive and counterfactual explanations.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 1) AI PROBLEM
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, a collection of 74 out of 113 (65.49%) original primary studies were found to formulate definitions for contfactual explanation and/or address theoretical accounts thereof (RQ 1 ).”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > E. DATA EXTRACTION AND SYNTHESIS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Thus, the outlined classification-oriented frameworks are evaluated on classifiers based on logistic regression [55], [136], [153], [158], decision trees [46], [80], [122], [140], [150], [155], [159], gradient boosted decision trees [147], support vector machines [131], [138], [146], random forests [81], [86], [142]-[144], neural networks [6], [48], [49], [91], [129], [130], [133], [135], [139], [”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 1) AI PROBLEM
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 205
- It may state two separate claims as one. 23
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Causal relations between the variables in explanation are not always considered binary (i.e., in the presence/absence of a cause).”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 1) CAUSAL CONTFACTUAL EXPLANATION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“107 within-query duplicates were identified and removed from further analysis.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > E. DATA EXTRACTION AND SYNTHESIS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, several researchers attempt to unify causal and non-causal contfactual explanation under the same paradigm.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. CONTFACTUAL EXPLANATION AS DEFINED IN RELATED THEORIES (ANSWER TO RQ 1 )
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“enforced us to discard 33 studies which did not satisfy the inclusion criteria.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > E. DATA EXTRACTION AND SYNTHESIS
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Contfactual explanations are used to justify automatic decisions obtained for a variety of AI-related problems.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 1) AI PROBLEM
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The analysis of the primary studies related to RQ 2 allows us to categorize the state-of-the-art contfactual explanation generation frameworks in accordance with the following criteria: (1) the problem the solution for which is to be explained (i.e., the AI problem); (2) the method employed to generate such an explanation (i.e., the explainability method); (3) the output representation of the expl”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > C. CONTFACTUAL EXPLANATIONS AS DEFINED IN AUTOMATIC GENERATION FRAMEWORKS (ANSWER TO RQ
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
None of the sources cited for this claim have been checked yet.
In the paper
“In three studies [67], [128], [137], the classifiers used in the experiments are not specified.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 1) AI PROBLEM
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A stronger impact of counterfactual explanation in the results of the joint query q 3 appears to affect significantly the overall allocation of the related keywords in the corresponding keyword map (see Fig. 7).”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > A. BIBLIOMETRIC ANALYSIS
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: Publication counts and author keyword maps are valid indicators of research attention and interdisciplinarity in the field.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Fig. 2, 75 publications were deemed irrelevant and filtered out at this stage.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > E. DATA EXTRACTION AND SYNTHESIS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similar notions are observed to be essential for contrastive explanation (see Fig. 6).”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > A. BIBLIOMETRIC ANALYSIS
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
- Rests on an assumption: Publication counts and author keyword maps are valid indicators of research attention and interdisciplinarity in the field.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Hence, a hybrid approach, originating from monism, 5 has been adopted to unify causal and non-causal counterfactual explanation.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) HYBRID CONTFACTUAL EXPLANATION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In contrast, other theoretical frameworks seek purely non-causal dependencies in explanation.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. CONTFACTUAL EXPLANATION AS DEFINED IN RELATED THEORIES (ANSWER TO RQ 1 )
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It is worth noting that several subcategorizations of causal contfactuals have been suggested within some of the aforementioned theoretical frameworks.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 1) CAUSAL CONTFACTUAL EXPLANATION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In the context of automatic decision-making, counterfactuals are found to be most generally defined as counterarguments for an alternative prediction [86].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) NON-CAUSAL CONTFACTUAL EXPLANATION
What the sources say
- Not checked yetcites [86]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“One of the classificationoriented frameworks [129] is extended to also handle the regression problem, i.e., learning a mapping function f from a training dataset X to a continuous output variable Y .”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 1) AI PROBLEM
What the sources say
- Not checked yetcites [129] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Finally, 19 papers were added to the review upon inspecting the bibliography of the primary studies.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > E. DATA EXTRACTION AND SYNTHESIS
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, a distinction between different clusters in the latter case is visible more clearly.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > A. BIBLIOMETRIC ANALYSIS
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: Publication counts and author keyword maps are valid indicators of research attention and interdisciplinarity in the field.
In the paper
“Causal contrastive explanation is frequently found to be designed as an answer to a why-question of the following canonical form: ''Why P rather than Q?'' where P is an explanandum (i.e., the fact to be explained), Q being a foil (i.e., one of alternative nonoccurring options) [21].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 1) CAUSAL CONTFACTUAL EXPLANATION
What the sources say
- Not checked yetcites [21] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: Counterfactual explanation encompasses contrastive, counterfactual, and contrastive-counterfactual explanation.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“All in all, 136 duplicates were removed.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > E. DATA EXTRACTION AND SYNTHESIS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In addition, 29 more duplicates were excluded when merging the sets of publications retrieved after q 4 and q 5 .”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > E. DATA EXTRACTION AND SYNTHESIS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“For instance, several explanation generators address computer vision tasks.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“The other frameworks addressing the regression problem aim to leverage gradientboosted decision trees [61] and indicate how large errors in regression tasks could be overcome [56].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 1) AI PROBLEM
What the sources say
- Not checked yetcites [61] source
- Not checked yetcites [56]
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“All the frameworks generating contfactual explanations can be classified by their explainability method as either modelspecific or model-agnostic.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Model-agnostic frameworks are largely found to use decision trees as part of the reasoning mechanism instead of explaining their output.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The explanation generation is considered a two-fold optimization problem of finding pertinent positives and negatives.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Depending on the problem considered, contfactuals are presented in the form of: (1) intervals or specific values of the appropriate feature values whose alteration would have changed the output (i.e., numerical or featurebased output);”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Numerical values (or intervals of values) associated to the most relevant features usually explain the behavior of AI algorithms.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“(2) single-or multiple-sentence coherent text (i.e., linguistic output);”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“(4) a multi-modal combination of (some of) the above (see Table 8).”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“(3) specific regions in the input image (i.e., visual output);”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“However, whereas certain publications do not provide any specific evaluation method, some do stress that human evaluation should be encouraged to estimate the quality and effectiveness of the generated counterfactuals [46], [150], [153].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 12.
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In computer science and AI, notions of counterfactual and contrastive explanation are found to be the most dissimilar when applied to non-overlapping problems.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 13.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“Furthermore, these changes are normally expected to be minimal [143].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 13.
What the sources say
- Not checked yetcites [143] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Both types of evaluation methods highlight the features supporting evidence as formulated in the contrastive explanation on the basis of the values that a perturbation variable takes on.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 12.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Two frameworks are evaluated in terms of both automatic metrics and human judgments.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 12.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In fact, definitions vary depending on domains of science and even approaches used for solving specific tasks.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > VI. CONCLUDING REMARKS
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The subjective methods include human preferences for certain types of contfactual explanation over others.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 12.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Contrastive and counterfactual explanations are found to be in great demand across various sub-fields of AI.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > VI. CONCLUDING REMARKS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Alternatively, only linguistic templates for such explanations are defined without any theoretical grounding in accordance with any accounts described in Section IV-B.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 13.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, the existing methodological differences affect greatly the definition of contfactual explanation found across various approaches.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > VI. CONCLUDING REMARKS
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Nevertheless, the corresponding line of research in AI makes little use of the rich theoretical background accumulated by now.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 13.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In general, a consensus among researchers has been observed on how contfactual explanations are defined irrespective of the theoretical framework proposed by individual authors.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 13.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Indeed, while the concept of ''the closest possible world'' does not always appear in the related publications, it turns out to be implicitly wired in almost all works.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 13.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In addition, the topic has recently attracted attention from researchers in XAI (see Fig. 13).”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > V. DISCUSSION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Nevertheless, the most commonly acceptable definition of a contfactual in the observed RQ 3 -related studies states that a contfactual explanation is a set of minimal feature modifications that makes the model change the prediction [81], [105], [122].”
- Also stated in: § VI. CONCLUDING REMARKS · plain
“the most commonly accepted definition of a contfactual explanation in the context of XAI refers to a minimal set of feature modifications that makes the model change the prediction.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. LINKS BETWEEN THEORETICAL AND PRACTICAL CONTRIBUTIONS TO CONTFACTUAL EXPLANATION GENERATION (ANSWER TO RQ 3 )
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- Also stated in: § VI. CONCLUDING REMARKS · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, the generated counterfactuals are inspected in terms of fairness which is calculated as the expected distance between the input and a counterfactual over distinct values for a specified feature set.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 12.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In image classification, it is found necessary to detect specific regions in the given test image.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 13.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“As a majority of the considered frameworks are designed for tackling classification problems, contfactual explanations operate on the notion of a contrast-class (e.g., see [155]) answering the question: ''How is the prediction altered when the observation changes, given a classifier and an observation?''”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 13.
What the sources say
- Not checked yetcites [155] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Finally, there is a number of marginal interpretations of contfactuals among the RQ 3 -related studies.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. LINKS BETWEEN THEORETICAL AND PRACTICAL CONTRIBUTIONS TO CONTFACTUAL EXPLANATION GENERATION (ANSWER TO RQ 3 )
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Despite the fact that the notion of contfactual explanation is found to be highly task-and domain-specific, the most commonly accepted definition of a contfactual explanation in the context of XAI refers to a minimal set of feature modifications that makes the model change the prediction.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > VI. CONCLUDING REMARKS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In humanities and social sciences, a major difference in various contfactual theories of explanation is observed to concern the causal nature of explanation and its extrapolation to non-causal cases.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 13.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“It is worth noting that the run-time of explanation generation algorithms is reported in addition to the evaluation metrics for several frameworks [132], [139], [146], [152], [156], [159].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 12.
What the sources say
- Not checked yetcites [132] source
- Not checked yetcites [139]
- Not checked yetcites [146], [152], [156], [159]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We find that a gap between philosophical accounts of contfactual explanation to scientific modeling and ML-related concepts makes the theoretical frameworks poorly applicable to XAI.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > VI. CONCLUDING REMARKS
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, these only include evaluations for factual explanations and are therefore excluded from the taxonomy group being discussed.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 12.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The rest of works redefine contfactual explanation on the basis of the problem-specific constraints without explicitly referring to the theoretical foundations described in Section IV-B.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. LINKS BETWEEN THEORETICAL AND PRACTICAL CONTRIBUTIONS TO CONTFACTUAL EXPLANATION GENERATION (ANSWER TO RQ 3 )
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Automatic metrics are complemented with a user study on intuitiveness, friendliness, comprehensibility, and understandability of generated explanations.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 12.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“More precisely, ML-specific contfactual explanations are designed to answer the question: ''Why was the outcome Y observed instead of Y ?''[148].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 13.
What the sources say
- Not checked yetcites [148]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Alternatively, Fang et al. [43] referred to contrastive reasoning as a procedure operating on ''butstatements'', as in ''all cars are polluting, but hybrid cars are not polluting'', which serves a principally different explanation generation task in comparison with the other aforementioned approaches.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > C. DISTINCTION BETWEEN CONTRASTIVE AND COUNTERFACTUAL EXPLANATION
What the sources say
- Not checked yetcites [43] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Thus, supporters of the pragmatic approach to explanation argue that it is exactly the ability to distinguish the answer to an explanatory question from a set of contrastive hypothesized alternatives that provides the explainee with sufficiently comprehensive information on the reasoning behind the question [13].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > I. INTRODUCTION
What the sources say
- Not checked yetcites [13] source
plaindirect- Asserted: fails — The authors may not put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it.
In the paper
“Counterfactuals describe events or states of the world that did not occur and implicitly or explicitly contradict factual world knowledge [27].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. COUNTERFACTUAL EXPLANATION
What the sources say
- Not checked yetcites [27] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“However, no agreement about a definition of explanation has been reached despite the fact that explanation has called a significant amount of attention in, e.g., philosophy of science [7], [8].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > I. INTRODUCTION
What the sources say
- Not checked yetcites [7]
- Not checked yetcites [8]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Thus, counterfactual reasoning in computer science is defined as the process of evaluating conditional claims about alternative possibilities and their consequences [33].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. COUNTERFACTUAL EXPLANATION
What the sources say
- Not checked yetcites [33] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“In addition, contrastive reasoning forms the basis of abductive inference [19], i.e., the process of inferring certain facts that render some observation plausible [20].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > I. INTRODUCTION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“In addition, counterfactual reasoning is claimed to be a key mechanism for explaining adaptive behavior in a changing environment [25], [26].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. COUNTERFACTUAL EXPLANATION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Moreover, counterfactuality is among the most fundamental concepts in theories of causation [36], [37].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. COUNTERFACTUAL EXPLANATION
What the sources say
- Not checked yetcites [36] source
- Not checked yetcites [37]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“This approach is also claimed to set a minimum criterion that an explanation must fulfill: it must favor the probability of the observed event P to all the hypothetical alternatives (Q 1 , Q 2 , . . ., Q n ) [14].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > I. INTRODUCTION
What the sources say
- Not checked yetcites [14] source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“The present survey has been undertaken as a systematic literature review following the guidelines by Kitchenham and Charters [58], Kitchenham et al. [59], and Wohlin [60].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > III. METHODOLOGY
What the sources say
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“The property of contrastiveness presupposes that an explanation answers the given why-question regarding the cause of the event in question (''Why did P happen?'') in terms of hypothesized non-occurring alternatives (''Why did P happen rather than Q?'') [12].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > I. INTRODUCTION
What the sources say
- Not checked yetcites [12] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Finally, the ability to explain a decision contrastively is claimed to lead to responsible decision-making [47].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. CONTRASTIVE AND COUNTERFACTUAL EXPLANATION IN THE CONTEXT OF XAI
What the sources say
- Not checked yetcites [47] source
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Their findings point to a strong correlation between the probability of the counterfactual conditional and causal strength judgments.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. COUNTERFACTUAL EXPLANATION
What the sources say
- Not checked yetcites [40]
boostedcorrelationaln = 6- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In contrast, McGill and Klein distinguished contrastive reasoning from its counterfactual counterpart [42].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > C. DISTINCTION BETWEEN CONTRASTIVE AND COUNTERFACTUAL EXPLANATION
What the sources say
- Not checked yetcites [42]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In this setting, the truth of a counterfactually inferred statement is resolved by: (1) modeling a situation where the smallest possible change in features of the actual world (as set in the antecedent) leads to a different (possibly, desired) state of things (the so-called ''closest'' or ''nearest'' possible world); and (2) estimating what is true in that setting [35].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. COUNTERFACTUAL EXPLANATION
What the sources say
- Not checked yetcites [35] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In this work we conduct a systematic literature review which provides readers with a thorough and reproducible analysis of the interdisciplinary research field under study.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Thus, Pearl extended the definition of the causal counterfactual to esti-mate the probability of the truth of the consequent caused by the antecedent (''a probability statement about the truth of y, had x been true, when it is known that y had been false when x was false'') [37].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. COUNTERFACTUAL EXPLANATION
What the sources say
- Not checked yetcites [37]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Thus, they proposed that the subjective probability of the counterfactual at the present time is the same as the conditional probability P(y|x) at some earlier time.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. COUNTERFACTUAL EXPLANATION
What the sources say
- Not checked yetcites [40]
plainn = 6- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The stochastic nature of predictions made by various AI algorithms is claimed to be among the main obstacles in reaching a true explanation [44].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. CONTRASTIVE AND COUNTERFACTUAL EXPLANATION IN THE CONTEXT OF XAI
What the sources say
- Not checked yetcites [44] source
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“On a similar note, Edgington regarded counterfactual judgments as uncertain conditional statements and therefore evaluated them by estimating their conditional probability given some endorsing event [41].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. COUNTERFACTUAL EXPLANATION
What the sources say
- Not checked yetcites [41]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Thus, contrastive explanations are claimed to be inherent to human cognition [16].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > I. INTRODUCTION
What the sources say
- Not checked yetcites [16]
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“A counterfactual definition of a cause of an arbitrary event traces back to Hume [38].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. COUNTERFACTUAL EXPLANATION
What the sources say
- Not checked yetcites [38] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“On the other hand, counterfactual explanations specify necessary minimal changes in the input so that a contrastive output is obtained.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. CONTRASTIVE AND COUNTERFACTUAL EXPLANATION IN THE CONTEXT OF XAI
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Indeed, counterfactuals are argued to represent a causal relation between the event happened in reality and its imaginary counterpart.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. COUNTERFACTUAL EXPLANATION
What the sources say
- Not checked yetcites [36] source
- Not checked yetcites [37]
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“All in all, counterfactual conditional statements are claimed to enable people to produce utterances that are factually false yet truthful irrespective of the interpretation adopted [32].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > B. COUNTERFACTUAL EXPLANATION
What the sources say
- Not checked yetcites [32] source
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In the context of AI, it often bases on judgments about why a certain outcome is predicted by an AI algorithm and hypotheses about causes with respect to given effects [10].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > I. INTRODUCTION
What the sources say
- Not checked yetcites [10] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“For instance, the need for explaining reasoning mechanisms behind such applications is now legally regulated in the European Union by means of the General Data Protection Regulation.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > I. INTRODUCTION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It is important to note that contrastive explanations point to the difference between the actual and a hypothetical decision.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. CONTRASTIVE AND COUNTERFACTUAL EXPLANATION IN THE CONTEXT OF XAI
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“As a result, the need to motivate automatic decisions with a clear explanation of why the algorithm outputs a particular decision has made the XAI research field grow quickly [5].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > I. INTRODUCTION
What the sources say
- Not checked yetcites [5] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In its most general form, explanation is normally treated as ''an answer to the question of why something is the case'' [9].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > I. INTRODUCTION
What the sources say
- Not checked yetcites [9]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“For instance, a counterfactual explanation of an ML-based algorithm prediction may describe ''the smallest change to the feature values that changes the prediction to a predefined output'' [53].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. CONTRASTIVE AND COUNTERFACTUAL EXPLANATION IN THE CONTEXT OF XAI
What the sources say
- Not checked yetcites [53] source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“However, these terms are sometimes used interchangeably in the context of XAI [48], [49].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. CONTRASTIVE AND COUNTERFACTUAL EXPLANATION IN THE CONTEXT OF XAI
What the sources say
- Not checked yetcites [48]
- Not checked yetcites [49] source
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Subsequently, we performed six queries over the title, abstract, and author keywords in the aforementioned libraries (see the overall structure of the query pipeline in Fig. 1).”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > C. SEARCH STRATEGY
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Subsequently, the preprocessed collection of publications was split into two overlapping subsets aiming to distinguish the publications covering theoretical accounts of contfactual explanation with the aim of extracting the related definitions, theories (or their inferences or implications) (q 4 ) and existing computational frameworks for contfactual explanation generation (q 5 ).”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > C. SEARCH STRATEGY
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In addition, we used a window span of three words (i.e., ''NEAR/3'') to ensure that the attributes ''counterfactual'' and ''contrastive'' relate to explanation.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > C. SEARCH STRATEGY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Counterfactual explanations are normally considered contrastive by nature and therefore present a source of valuable complementary information to a given automatic prediction [52].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. CONTRASTIVE AND COUNTERFACTUAL EXPLANATION IN THE CONTEXT OF XAI
What the sources say
- Not checked yetcites [52] source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, publications obtained as a result of q 5 address RQ 2 .”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > C. SEARCH STRATEGY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“Furthermore, Keane et al. argue that applying case-based reasoning techniques for generating counterfactuals increases their explanatory competence [51].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. CONTRASTIVE AND COUNTERFACTUAL EXPLANATION IN THE CONTEXT OF XAI
What the sources say
- Not checked yetcites [51]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, we excluded duplicate reports of the same studies appeared in both Scopus and WoS.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. INCLUSION AND EXCLUSION CRITERIA
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“As for the publications related to RQ 2 , we discarded: (1) the publications which were not related to AI algorithms or applications as well as (2) those where the proposed framework did not provide any human-comprehensible contfactual explanations as output.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. INCLUSION AND EXCLUSION CRITERIA
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“As for the publications related to RQ 1 , we also removed: (1) the studies whose contents did not introduce any contfactual theory of explanation or (2) those containing no formal or informal definition of contrastive or counterfactual or contrastivecounterfactual explanation.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. INCLUSION AND EXCLUSION CRITERIA
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In order to capture existing computational frameworks for contfactual explanation generation, we included publications that: (1) present a novel approach, method, or framework for contfactual explanation generation whose output can serve to explain the reasoning of an AI algorithm and (2) are found in such subject areas as computer science, mathematics, engineering as well as in their sub-fields.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. INCLUSION AND EXCLUSION CRITERIA
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“The search web tools of the selected digital libraries allow researchers to reproduce the original study.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > C. SEARCH STRATEGY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We selected the digital libraries Scopus and Web of Science (WoS) to retrieve relevant publications from.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > C. SEARCH STRATEGY
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“In the context of XAI, an explanation for an automatic decision or prediction, treated as an observation, can be obtained abductively by attempting the search problem over the set of the known information concerning that observation [50].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. CONTRASTIVE AND COUNTERFACTUAL EXPLANATION IN THE CONTEXT OF XAI
What the sources say
- Not checked yetcites [50] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“It is worth noting that the number of publications in 2020 is limited to the search date.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > A. BIBLIOMETRIC ANALYSIS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Queries q 1 and q 2 embrace all the up-to-date publications containing mentions of counterfactual and contrastive explanation, respectively, found across all subject areas.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > C. SEARCH STRATEGY
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These libraries do not only include research publications in computing but also index studies across all scientific fields, which allows for an objective analysis of the interdisciplinary literature relevant to the research questions posed.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > C. SEARCH STRATEGY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“In order to capture all relevant publications, we only used the corresponding word-stems to allow for maximal diversity of the retrieved papers.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > C. SEARCH STRATEGY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Indeed, contfactuals are particularly suitable for informing the end-user why a given data example is assigned a particular class label.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 1) AI PROBLEM
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“It is important to note that publications retrieved as a result of q 4 form an exhaustive set of papers addressing RQ 1 .”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > C. SEARCH STRATEGY
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
In the paper
“Alternatively, counterfactual explanation is widely addressed in the paradigm of case-based reasoning, i.e., a family of problem solving methods based on appeals to precedent solutions.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > D. CONTRASTIVE AND COUNTERFACTUAL EXPLANATION IN THE CONTEXT OF XAI
What the sources say
- Not checked yetcites [14] source
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The keywords identified in the studies related to q 3 testify that the issue of contfactual explanation is highly interdisciplinary and finds application in both humanities and natural sciences.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > A. BIBLIOMETRIC ANALYSIS
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Publication counts and author keyword maps are valid indicators of research attention and interdisciplinarity in the field.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Finally, the papers that q 6 returned address RQ 3 .”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > C. SEARCH STRATEGY
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, their use guarantees performing equivalent queries across both libraries.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > C. SEARCH STRATEGY
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“Subsequently, the generated counterfactuals are ranked using a XOR-based distance to find the most relevant counterfactual pertinent to the given contrast class.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [122] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Russell et al. adopt a similar approach to encompass continuous and discrete variables as well as the combination of the two [153].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [153] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“It is worth noting that the framework is not designed to provide explanations for ML algorithms.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 1) AI PROBLEM
What the sources say
- Not checked yetcites [93]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The best counterfactual explanation is then selected to be the most class-specific to the counterfactual image while being the most relevant to the input image.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [86]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“This method is further extended to generating counterfactuals for fuzzy decision trees.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [122] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“For instance, Fernández et al. [80] present a recursive algorithm which extracts counterfactuals in the form of contrast-class decision tree nodes.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [80] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Hendricks et al. bind an input visual image with a paired textual counterfactual explanation generated by a recurrent neural network [86].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [86]
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“They obtain a counterfactual explanation by removing a minimal set of user actions so that the output recommendation changes.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > Ghazimatin et al.
What the sources say
- Not checked yetcites [84] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In order to guarantee the coherence of the counterfactual data example used for explanation, an integer programming-based method is proposed.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [153] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In addition, the generated counterfactual explanations are claimed to be diverse, as diversity constraints are applied iteratively to a set of candidate counterfactuals.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [153] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Mosca et al. introduce an argumentation-based framework for social network management [149].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > Ghazimatin et al.
What the sources say
- Not checked yetcites [149] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The relevance of the generated counterfactuals is then measured by calculating a variant of the Gower distance.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [80] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The researchers make use of the weighted Euclidean distance to present the forecast as a contrast to the neighbouring vectorized promotions.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [61] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The first of the considered frameworks (in chronological order) [93] offers explanations by reasoning abductively over the information extracted from a given knowledge base to answer a specific contrastive question.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 1) AI PROBLEM
What the sources say
- Not checked yetcites [93]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The explanations based on such graphs are passed on to human subjects to evaluate the contrastive answers to the predefined questions.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [128] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Thus, Ustun et al. tackle the problem of actionability, i.e., constraining the generated counterfactuals in such a manner that the imposed changes ''do not alter immutable features'' and that they ''do not alter mutable features in an infeasible way'' [158].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [158] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“A counterfactual explanation is, in this case, dependent on how likely a selected region in the given frame is classified positive and not negative, hence all such regions are scored and normalized.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [91] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Designed specifically for decision trees, their ''Glass-box'' frame- work is argued to be easily extendable to capture the output of other logical (rule-based) models.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In this setting, an explanation is considered to be a consistent set of disjunctive literals for the explanation-seeking question.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 1) AI PROBLEM
What the sources say
- Not checked yetcites [93]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Ghazimatin et al. propose a graph-based recommendation system in a counterfactual setup [84].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > Ghazimatin et al.
What the sources say
- Not checked yetcites [84] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In their framework, a number of candidate explanations (both image-relevant and non-relevant) are generated, paired, and ranked.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [86]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“To approach this problem, a mixed integer programming method is employed.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [158] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The proposed metric penalizes the number of feature changes when traversing the tree so that sparsity is promoted.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [80] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Alternatively, Kanehira et al. train a post-hoc explanatory model to justify a video classifier's output [91].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [91] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Aguilar-Palacios et al. generate contrastive explanations using gradient boosted decision trees to forecast promotional sales [61].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [61] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Alternatively, Sokol and Flash rely on the Manhattan distance measuring leaf-to-leaf distance in the tree to retrieve the most relevant counterfactuals [46], [155].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“They use contrastive explanations to answer critical questions about agent actions in the context of multi-user privacy conflict.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > Ghazimatin et al.
What the sources say
- Not checked yetcites [149] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“For instance, Akula et al. [128] present an empirical study where input video frames are paired with the corresponding AND-OR graphs, i.e., compositional recursively defined graphbased knowledge representations capturing contextual information.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [128] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Stepin et al. retrieve counterfactuals from a rule matrix where each rule is encoded in terms of all possible feature values [122].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [122] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“However, this framework is limited to: (1) explaining predictions of only linear classifiers and (2) a simple structure of the textual explanation template.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [153] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“However, the continuous output is, in this case, subsequently converted to a lower-scale discrete value mapped to a textual description similar to that typical of a classification problem.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 1) AI PROBLEM
What the sources say
- Not checked yetcites [129] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Counterfactuals are selected from a local neighbourhood circling around the test example with the radius of the distance to the closest correctly predicted data point of a contrast-class.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [144] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Labaien et al. generate contrastive explanations for time-series data [141].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [141] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Anjomshoae et al. [129] focus on contrastive explanations that maximize contextual importance and contextual utility.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [129] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“A similar limitation is observed due to possibly high variability of the input.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [129] source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“As plan explicability is reformulated in terms of its comprehensibility by an end-user.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [132] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Conversely, Sreedharan et al. consider the task of automatic analysis of counterfactual explanations in their ''Hierarchical Expertise-Level Modeling'' framework [156].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [156] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The problem of obtaining contrastive explanations is designed as a Bayesian inference problem, with the posterior distribution to be maximized defined as the probability of a contrastive explanation given a set of positive and negative plan traces.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [94] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Similarly, van der Waa et al. randomly sample or generate a data set in the neighbourhood local to the data point in question [159].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [159] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Hence, their objective is to find an optimal policy ''that minimizes the expected cumulative cost of reaching a goal state over all closed policies''.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [157] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The researchers attempt to find the best counterfactual explanation by minimizing the number of attributes changed.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [148]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“They argue that a synthesized counterfactual data point must be connected to the training data.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [144] source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Hence, they suggest the model-agnostic ''Search for Explanations for Document Classification'' algorithm for retrieving counterfactual explanations.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [146] source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Pawelczyk et al. make use of an autoencoder architecture for a pretrained classifier performing counterfactual search in the nearest neighbor style [151].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [151] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In addition, Zhao and Sukkerd explain an autonomous system's behaviour modeling it as a Markov decision process [161].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [161]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“A decision tree is then trained to select the foil based on the minimum number of nodes between the original data point and the candidate foils.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [159] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“However, it is only directly applicable to binary linear classifiers, whereas heuristics are proposed for nonlinear models.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [146] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Fixing the optimal set of weights of a trained classifier, the objective function minimizes the distance between the nearest data points of opposing classes.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [6] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In contrast, Martens and Provost argue that decision trees are an inadequate tool for representing, e.g., large documents [146].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [146] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In their ''Local Rule-based Explanation'' framework, they generate a local neighbourhood for the given pre-classified data example using a genetic algorithm and subsequently train a decision tree on that newly obtained dataset to select a minimally distant foil within that local neighbourhood.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [140] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The explanation is believed to justify the rejection of the policies alternative to the optimal one.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [157] source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Furthermore, the gradient direction is constrained to ensure the ethical adequacy of the explanation generated.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [148]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Thus, the context-based values are calculated for each feature used by a black-box model observing the changes in the output as the input varies across the range of all possible input values.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [129] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Thus, a contrastive explanation is presented as a product of the analysis of the optimal policy at the next time step and an opposing policy on the basis of the objective values.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [161]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Subsequently, the generated contrastive explanations are attributed to a specific group of users by means of ontology engineering at the cognitive level of the framework to make the explanations adaptive.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [150] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In accordance with the findings in the previous Section IV-C1, contfactual explanations have a great potential for automatic planning-related tasks.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“Being based on model-agnostic and problem-independent concepts, this framework is shown to be universally applicable to various classification and regression algorithms.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [129] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“For instance, Sukkerd et al. formulate the planning problem as the shortest stochastic path problem and develop the corresponding problem solver to obtain a contrastive explanation [157].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [157] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The researchers suggest the use of the Manhattan distance weighted by the inverse median absolute deviation to calculate the proximity of a counterfactual to the input data example.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [6] source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“For instance, Kim et al. employ a Bayesian probabilistic model for generating contrastive explanations [94].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [94] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Laugel et al. raise the issue of justification for counterfactual explanation [144].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [144] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Thus, the framework operates on a pair of plan traces defined in terms of linear temporal logic templates.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [94] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The robot's model is adapted to the updates of human's model of the problem.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [132] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Adversarial examples that could serve as the basis for the counterfactual explanation of the output of deep learning models are searched for with the aim of minimizing the loss with respect to the attributes (features) between the original and counterfactual data examples.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [148]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In addition, Chakraborti et al. formulate the multi-model planning problem as a tuple consisting of the planner's model of the problem and the corresponding human approximation thereof [132].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [132] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“In contrast, multiple-sentence explanations provide end-users with specific details for the given decision [153], [156], [157], [161] or explain multiple decisions at once [146].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
What the sources say
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Fernández et al. introduce the notion of counterfactual sets to enhance counterfactual diversity [81].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [81] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“and allow for dialogic interaction [150].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
What the sources say
- Not checked yetcites [150] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“or explained by contrastive patterns between the given data example and that of an opposing class in one iteration [130], [144] or a series thereof [49]”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
What the sources say
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“They also answer end-user's inquiries with respect to the automatic decision in question [128], [159].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“They can be extracted from interpretable featurevalue pairs as a result of pruning in the search space [132].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
What the sources say
- Not checked yetcites [132] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The candidate counterfactuals are then clustered, as the initial local neighbourhood is updated to become a more extensive hyperspherical layer, until it can no longer be extended.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [144] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Such explanations suggest featurevalue based instructions [48], [122], [131] or alternative actions for a possible output change [84], [149].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
What the sources say
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“They explain random forest predictions by fusing different tree predictors so that the resulting counterfactual set contains the most relevant counterfactual.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [81] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In addition, visual-linguistic explanations identify counterfactuality in videos [91]”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
What the sources say
- Not checked yetcites [91] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“It is worth noting that counterfactuals are presented in the form of conditional distributions and not structural equations despite the fact that the causal model employed follows Pearl's formalism [37].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [55]
- Not checked yetcites [37]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“by depicting critical regions absent in the input data example that determine what lacks in the image to be classified differently [135]”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
What the sources say
- Not checked yetcites [135] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The other neighboring counterfactuals serve to diversify the output explanation.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [81] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“While a textual explanation summarizes the degree of importance of the selected features, a visual explanation may present contextual in-method metrics that justify the classifier's reasoning [129].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
What the sources say
- Not checked yetcites [129] source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“They employ a generative model to construct a saliency map while masking the other regions of the input image.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [133] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In computer vision-related classification tasks, Chang et al. find the smallest region in the image whose substitution would change the classifier's prediction [133].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [133] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Contfactual explanations, as a mixture of tabular and visual output representations, appear also in an augmented reality framework [137].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
What the sources say
- Not checked yetcites [137] source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Single-sentence textual explanations combine a textual description with explicitly stated numerical feature values [6].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
What the sources say
- Not checked yetcites [6] source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Kusner et al. propose a causal model to assess the so-called ''counterfactual fairness'' [55].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [55]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“On the other hand, visual input data (i.e., images) are associated with saliency maps [133]”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
What the sources say
- Not checked yetcites [133] source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“They argue that the distance from the test instance to a counterfactual does not sufficiently measure counterfactual's relevance, as the counterfactual in question may appear disconnected from the ground-truth data.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [100] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Laugel et al. [100] enhance the work on justified counterfactual explanations.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 2) EXPLAINABILITY METHOD
What the sources say
- Not checked yetcites [100] source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“or by visualizing spatial regions associated to data examples of opposed classes [139].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
What the sources say
- Not checked yetcites [139]
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“For instance, it is common to distinguish between intrinsic and extrinsic evaluation methods in natural language generation [163].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 4) EVALUATION METHOD
What the sources say
- Not checked yetcites [163] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In addition, Gatt and Krahmer make a distinction between ''objective'' (automatic, corpus-based) and ''subjective'' (human judgements) metrics [164].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 4) EVALUATION METHOD
What the sources say
- Not checked yetcites [164] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thanks to their methodological universality, they can be extrapolated to other (non-linguistic) modalities of generated explanations (e.g., numerical, visual, or multi-modal) and are therefore used to form the basis of the evaluation method classification in this review.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 12.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As most of the current ML-tasks are centered around singling meaningful patterns out from unstructured data, establishing causal relations appears to be among the AI problems that are yet to attract global attention.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 13.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Nevertheless, explanations of different modalities are not necessarily merged.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Finally, interaction with users can be enhanced by means of voice-based explanations combined with textual explanations [46], [155].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > 3) OUTPUT REPRESENTATION
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Several metrics can be extended to be applied to other approaches.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 12.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“While some rule-based approaches used in expert systems are justified theoretically (e.g., see [93]), newly emerging tasks present novel challenges for theorists and call for updating the theories developed so far.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 13.
What the sources say
- Not checked yetcites [93]
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Promoting interactivity (e.g., by engaging the end-user to participate in an explanatory dialogue with the system) is expected to make explanation social and further increase user's trust in the system's output.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 13.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Mainly applied to a wide range of tasks in computer vision and natural language processing, they present a powerful tool that enhances human-machine interaction and allows for further personalization of the output generated by various AI algorithms, including ML-based black-box algorithms.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > VI. CONCLUDING REMARKS
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“This research methodology allowed us to carry out an unbiased reproducible study from an interdisciplinary topic-specific search in reputable and trustworthy sources.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > VI. CONCLUDING REMARKS
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the amount of publications for RQ 3 may be somewhat misleading, as contfactual explanations are often redefined without specifically referring to theoretical contributors in explanation.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 13.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“The consistency of output explanations is otherwise shown by measuring their accuracy on the basis of mean error, mean absolute error, and mean absolute percentage error [61].”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 12.
What the sources say
- Not checked yetcites [61] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper relies on this without support.
There is a gap between theoretical approaches to contrastive and counterfactual explanation and the corresponding computational frameworks.
“It is of crucial importance to bridge the gap between theoretical approaches to contrastive and counterfactual explanation and the corresponding computational frameworks.”
The paper relies on this without support.
Publication counts and author keyword maps are valid indicators of research attention and interdisciplinarity in the field.
“The bibliometric analysis over the queries q 1 , q 2 , and q 3 allows us to obtain a big picture of the research area of contfactual explanation generation and spot its key characteristics.”
“Author keyword maps allow us to present an overview of the terms most relevant to those specified in the preparatory queries (q 1 , q 2 , and q 3 ).”
The paper relies on this without support.
Counterfactual explanation encompasses contrastive, counterfactual, and contrastive-counterfactual explanation.
“Remind that contfactual explanation embraces contrastive, counterfactual, and contrastive-counterfactual explanation.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, explainability is a growing issue for financial institutions who seek to ensure that AI conclusions and procedures are ''easily comprehensible'' by bank officials.”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [51]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Since all of the input variables have to be easy to comprehend, articulating all of the system's components and operations is a barrier when trying to break the system down into its constituent aspects [45], [51].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [51]
Experimental — accuracy not yet measuredIn the citing paper
“In the work of Stepin et al. [141] counterfactual explanations offer another avenue for achieving explanatory depth in XAI by providing insights into”
From Recent Emerging Techniques in Explainable Artificial Intelligence to Enhance the Interpretable and Understanding of AI Models for Human (2025) · cites this paper as [141]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 12.
“It is worth noting that all the considered frameworks are evaluated by means of intrinsic (either subjective or objective) metrics.”
§ A Survey of Contrastive and Counterfactual Explanation Generation Methods for Explainable Artificial Intelligence > FIGURE 12.
“17 out of 52 (32.69%) of the considered publications do not evaluate their frameworks, i.e., neither automatic metrics for contfactual explanation generation are suggested nor a human evaluation survey is presented in such publications.”
Citations
Cites
Software survey: VOSviewer, a computer program for bibliometric mapping
van Eck, N. J., & Waltman, L. (2010) · Scientometrics · doi:10.1007/s11192-009-0146-3 · Cited by 21,484 · cited 1×
“In addition, we utilized the tool VOSViewer [162] to generate the author keyword maps.”
Method"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · cited 1×
“the ability to explain such automatic decisions undermines users’ trust and hence decreases usability of such systems [2].”
BackgroundRudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 1×
“In addition, many of the existing eXplainable AI (XAI 1 ) methods provide summaries of automatically made predictions rather than true explanations [4].”
BackgroundBarredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · cited 1×
“and accountable (ensuring responsibility for the explanations provided) [57].”
BackgroundExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
CAUSALITY: MODELS, REASONING AND INFERENCE
Hitchcock, C., & Pearl, J. (2001) · The Philosophical Review · doi:10.2307/3182612 · Cited by 4,827
Systematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537 · cited 1×
“The present survey has been undertaken as a systematic literature review following the guidelines by Kitchenham and Charters [58], Kitchenham et al. [59], and Wohlin [60].”
MethodWohlin, C. (2014) · Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2601248.2601268 · Cited by 4,234 · cited 2×
“The present survey has been undertaken as a systematic literature review following the guidelines by Kitchenham and Charters [58], Kitchenham et al. [59], and Wohlin [60].”
BackgroundMethodMaking things happen: a theory of causal explanation
(2004) · Choice Reviews Online · doi:10.5860/choice.41-5840 · Cited by 3,383
Causality: models, reasoning, and inference
(2000) · Choice Reviews Online · doi:10.5860/choice.38-1494a · Cited by 2,968
Research works printed but not linked: 3
- lewis, 1973, Counterfactuals
- lewis, 1986, On the Plurality of Worlds
- molnar, 2019, Interpretable Machine Learning A Guide for Making Black Box Models Explainable
Cited by
Sorted by citations
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Self-citationExplainable AI (XAI): A systematic meta-survey of current challenges and future opportunities
Saeed, W., & Omlin, C. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110273 · Cited by 783
Survey of Explainable AI Techniques in Healthcare
Chaddad, A., Peng, J., Xu, J., & Bouridane, A. (2023) · Sensors · doi:10.3390/s23020634 · Cited by 591
“, offer explanations from various perspectives, so that users have the option to choose explanations), and (3) counterfactual explanations to enhance human–computer interactions and produces personalized output with the AI model [105].”
BackgroundNauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schlötterer, J., van Keulen, M., & Seifert, C. (2023) · ACM Computing Surveys · doi:10.1145/3583558 · Cited by 590
“[239] discuss XAI methods and their evaluation for contrastive and counterfactual explanations, Tintarev and Masthoff [250] and Nunes and Jannach [182] discuss explanations and their evaluation for recommender systems, and Markus et al.”
Counterfactual explanations and how to find them: literature review and benchmarking
Guidotti, R. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00831-6 · Cited by 583
A Review of Trustworthy and Explainable Artificial Intelligence (XAI)
Chamola, V., Hassija, V., Sulthana, A. R., Ghosh, D., Dhingra, D., & Sikdar, B. (2023) · IEEE Access · doi:10.1109/access.2023.3294569 · Cited by 352
“Various client advocacy organizations, competitors, and internal stakeholders within banking firms all exhibit this upsurge [1], [51].”
BackgroundMethodSchwalbe, G., & Finzel, B. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00867-8 · Cited by 422
Nazar, M., Alam, M. M., Yafi, E., & Su'ud, M. M. (2021) · IEEE Access · doi:10.1109/access.2021.3127881 · Cited by 312
Chou, Y.-L., Moreira, C., Bruza, P., Ouyang, C., & Jorge, J. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.11.003 · Cited by 302
Kute, D. V., Pradhan, B., Shukla, N., & Alamri, A. (2021) · IEEE Access · doi:10.1109/access.2021.3086230 · Cited by 209
“[96] examined the theoretical foundation for C&C explanations and reported the state-of-the-art computation frameworks for generating the C&C explanations and identi-”
BackgroundPre-Trained Language Models for Text Generation: A Survey
Li, J., Tang, T., Zhao, W. X., Nie, J., & Wen, J.-R. (2024) · ACM Computing Surveys · doi:10.1145/3649449 · Cited by 383
“Explanation generation is designed to interpret AI algorithms, elucidating how models arrive at specific decisions to enhance users’ trust and augment the usability of the algorithms [221].”
MethodDing, W., Abdel‐Basset, M., Hawash, H., & Ali, A. M. A. (2022) · Information Sciences · doi:10.1016/j.ins.2022.10.013 · Cited by 214
Singhal, A., Neveditsin, N., Tanveer, H., & Mago, V. (2024) · JMIR Medical Informatics · doi:10.2196/50048 · Cited by 172
“…models [86]; detailed discussions of model interpretability will follow in a dedicated subsection); (3) explanation generation—this technique produces explanations for a model’s outputs, offering insights into its decision-making process through visualizations or natural language descriptions [87].”
MethodExplainable AI Over the Internet of Things (IoT): Overview, State-of-the-Art and Future Directions
Jagatheesaperumal, S. K., Pham, Q., Ruby, R., Yang, Z., Xu, C., & Zhang, Z. (2022) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2022.3215676 · Cited by 198
“[43] XAI models An overview on the contrastive and counterfactual explanation generation methods for XAI.”
A Survey of Algorithmic Recourse: Contrastive Explanations and Consequential Recommendations
Karimi, A.-H., Barthe, G., Schölkopf, B., & Valera, I. (2023) · ACM Computing Surveys · doi:10.1145/3527848 · Cited by 301
“Next, we summarize the technical literature in Table 1 (subsequent surveys can be found else-where [117, 215, 227]).”
BackgroundMathew, D. E., Ebem, D. U., Ikegwu, A. C., Ukeoma, P. E., & Dibiaezue, N. F. (2025) · Neural Processing Letters · doi:10.1007/s11063-025-11732-2 · Cited by 168
“In the work of Stepin et al. [141] counterfactual explanations offer another avenue for achieving explanatory depth in XAI by providing insights into “what-if” scenarios by generating hypothetical instances where model predictions change, users can gain a better understanding of the decision…”
Mehrotra, S., Degachi, C., Vereschak, O., Jonker, C. M., & Tielman, M. L. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3696449 · Cited by 219
The coming of age of interpretable and explainable machine learning models
Lisboa, P. J. G., Saralajew, S., Vellido, A., Fernández-Domenech, R., & Villmann, T. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.02.040 · Cited by 167
Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review
Kuznietsov, A., Gyevnár, B., Wang, C., Peters, S., & Albrecht, S. V. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3474469 · Cited by 206
“To comprehensively explore the published methods, we have decided to perform a systematic literature review following the recommendations of Kitchenham and Charters [137] and the review methodology section of Stepin et al. [138].”
MethodModel agnostic generation of counterfactual explanations for molecules
Wellawatte, G. P., Seshadri, A., & White, A. D. (2022) · Chemical Science · doi:10.1039/d1sc05259d · Cited by 143
EXplainable Artificial Intelligence (XAI)—From Theory to Methods and Applications
Ortigossa, E. S., Gonçalves, T., & Nonato, L. G. (2024) · IEEE Access · doi:10.1109/access.2024.3409843 · Cited by 124
“That is the reason why XAI literature often uses term ‘‘contfactual’’ to agglutinating explanation solutions covering both concepts independently and their fusion [122].”
BackgroundInfluentialOn generating trustworthy counterfactual explanations
Del Ser, J., Barredo-Arrieta, A., Díaz-Rodríguez, N., Herrera, F., Saranti, A., & Holzinger, A. (2024) · Information Sciences · doi:10.1016/j.ins.2023.119898 · Cited by 122
Houssein, E. H., Gamal, A. M., Younis, E. M. G., & Mohamed, E. (2025) · Cluster Computing · doi:10.1007/s10586-025-05281-5 · Cited by 148
Mylrea, M., & Robinson, N. (2023) · Entropy · doi:10.3390/e25101429 · Cited by 127
Explainability and causability in digital pathology
Plass, M., Kargl, M., Kiehl, T., Regitnig, P., Geißler, C., Evans, T., Zerbe, N., Carvalho, R., Holzinger, A., & Müller, H. (2023) · The Journal of Pathology Clinical Research · doi:10.1002/cjp2.322 · Cited by 99
“Counterfactuals aim to reveal necessary minimal changes in the input so that one would obtain a contrastive output of the ML model [30,35].”
BackgroundCabitza, F., Campagner, A., Malgieri, G., Natali, C., Schneeberger, D., Stoeger, K., & Holzinger, A. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.118888 · Cited by 89
Explainable Machine Learning for Property Predictions in Compound Optimization
Rodríguez-Pérez, R., & Bajorath, J. (2021) · Journal of Medicinal Chemistry · doi:10.1021/acs.jmedchem.1c01789 · Cited by 88
Contrastive Explanations for Model Interpretability
Jacovi, A., Swayamdipta, S., Ravfogel, S., Elazar, Y., Choi, Y., & Goldberg, Y. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.120 · Cited by 83
Nor, A. K. M., Pedapati, S. R., Muhammad, M. B., & Leiva, V. (2021) · Sensors · doi:10.3390/s21238020 · Cited by 86
“(iv) Challenges in PHM-XAI progress: Crosschecks were done with the general challenges raised in [14,32,34,38] associated with:”
BackgroundSalvi, M., Seoni, S., Campagner, A., Gertych, A., Acharya, U. R., Molinari, F., & Cabitza, F. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105846 · Cited by 89
Counterfactual Shapley Additive Explanations
Albini, E., Long, J., Dervovic, D., & Magazzeni, D. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533168 · Cited by 86
“, [54]) among other methods (we refer the reader to [31, 58, 28, 64] for recent surveys).”
BackgroundMartins, T., de Almeida, A., Cardoso, E., & Nunes, L. (2023) · IEEE Access · doi:10.1109/access.2023.3347028 · Cited by 75
Niri, M. F., Aslansefat, K., Haghi, S., Hashemian, M., Daub, R., & Marco, J. (2023) · Energies · doi:10.3390/en16176360 · Cited by 62
The Intriguing Relation Between Counterfactual Explanations and Adversarial Examples
Freiesleben, T. (2021) · Minds and Machines · doi:10.1007/s11023-021-09580-9 · Cited by 76
“However, due to the great number of papers on both fields (Yuan et al., 2019; Verma et al., 2020; Serban et al., 2020; Stepin et al., 2021) our framework will probably not be able to cover all usages.”
BackgroundTowards Data-And Knowledge-Driven AI: A Survey on Neuro-Symbolic Computing
Wang, W., Yang, Y., & Wu, F. Q. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3483273 · Cited by 126
“However, there are only a few models [148], [149], [150] can be barely recognized as Type 6.”
Interpretable and explainable predictive machine learning models for data-driven protein engineering
Medina-Ortiz, D., Khalifeh, A., Anvari-Kazemabad, H., & Davari, M. D. (2024) · Biotechnology Advances · doi:10.1016/j.biotechadv.2024.108495 · Cited by 55
The Dawn of the Human-Machine Era: A forecast of new and emerging language technologies
Sayers, D., Sousa‐Silva, R., Höhn, S., Ahmedi, L., Allkivi-Metsoja, K., Anastasiou, D., Beňuš, Š., Bowker, L., Bytyçi, E., Catalá, A., et al. (2021) · doi:10.17011/jyx/reports/20210518/1 · Cited by 64
Counterfactual Explanations Can Be ManipulatedPreprint
Slack, D., Hilgard, S., Lakkaraju, H., & Singh, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02666 · Cited by 58
“These counterfactual explanations are widely used to compute recourse and assess the fairness of models [3, 19, 20].”
BackgroundBridging the Transparency Gap: What Can Explainable AI Learn from the AI Act?
Gyevnár, B., Ferguson, N., & Schäfer, B. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230367 · Cited by 66
Alfeo, A. L., Zippo, A. G., Catrambone, V., Cimino, M. G. C. A., Toschi, N., & Valenza, G. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107550 · Cited by 38
Wilson, A., Stefanik, C., & Shank, D. B. (2022) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2022.100229 · Cited by 38
Interpreting Language Models with Contrastive Explanations
Yin, K., & Neubig, G. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.14 · Cited by 78
The Robustness of Counterfactual Explanations Over Time
Ferrario, A., & Loi, M. (2022) · IEEE Access · doi:10.1109/access.2022.3196917 · Cited by 47
R e LAX: Reinforcement Learning Agent Explainer for Arbitrary Predictive Models
Chen, Z., Silvestri, F., Wang, J., Zhu, H., Ahn, H., & Tolomei, G. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557429 · Cited by 51
“In this work, we focus on specific post-hoc explanations called counterfactual explanations , which are used to interpret predictions of individual instances in the form: “ If A had been different, B would not have occurred ” [37][43].”
MethodCARLA: A Python Library to Benchmark Algorithmic Recourse and Counterfactual Explanation AlgorithmsPreprint
Pawelczyk, M., Bielawski, S., van-den-Heuvel, J., Richter, T., & Kasneci, G. (2021) · NeurIPS Datasets and Benchmarks · doi:10.48550/arxiv.2108.00783 · Cited by 137
“As documented in recent reviews, there exists a quickly growing literature with available methods (see Figure 1 and [54, 24, 60]), reflecting the insight that the understanding of complex machine learning models is an elementary ingredient for a wide and safe technology adoption.”
BackgroundNguyen, V. G., Sharma, P., Ağbulut, Ü., Le, H. S., Cao, D. N., Dzida, M., Osman, S. M., Le, H. C., & Trần, V. D. (2024) · International Journal of Green Energy · doi:10.1080/15435075.2024.2326076 · Cited by 36
“XAI is a collection of techniques that are designed to eliminate the layers of complexity that are inherent in modern machine learning models (Stepin et al. 2021).”
Shang, R., Feng, K. J. K., & Shah, C. S. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533189 · Cited by 42
“In the following narration, we will collapse the definition of contrastive and counterfactual explanations as done by many other researchers [41].”
BackgroundResultInfluentialEffects of explainable artificial intelligence in neurology decision support
Gombolay, G. Y., Silva, A., Schrum, M., Gopalan, N., Hallman‐Cooper, J., Dutt, M., & Gombolay, M. (2024) · Annals of Clinical and Translational Neurology · doi:10.1002/acn3.52036 · Cited by 39
“The field of explainable artificial intelligence 3 (xAI) has produced numerous methods for inspecting AI models, for example, feature importance, 4 counterfactual reasoning, 5 and decision trees.”
MethodBorrego-Díaz, J., & Galán-Páez, J. (2022) · Minds and Machines · doi:10.1007/s11023-022-09603-z · Cited by 34
“«64» Even using intelligible models for the stakeholder, it is likely that the process of explaining remains being an interactive and contrasting task, something deeply analyzed in XAI (Stepin et al., 2021).”
BackgroundOn the robustness of sparse counterfactual explanations to adverse perturbations
Virgolin, M., & Fracaros, S. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2022.103840 · Cited by 40
“For recent surveys on counterfactual explanations, the reader is referred to [56, 16, 57].”
BackgroundZhang, Y., Song, S., Li, X., Gao, S., & Raubal, M. (2024) · Sustainable Cities and Society · doi:10.1016/j.scs.2024.105739 · Cited by 33
Gerlach, J., Hoppe, P. E., Jagels, S., Licker, L., & Breitner, M. H. (2022) · Electronic Markets · doi:10.1007/s12525-022-00603-6 · Cited by 35
Feature-Weighted Counterfactual-Based Explanation for Bankruptcy Prediction
Cho, S. H., & Shin, K.-S. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.119390 · Cited by 38
Explainable Machine Learning with Prior Knowledge: An Overview
Beckh, K., Müller, S., Jakobs, M., Toborek, V., Tan, H., Fischer, R., Welke, P., Houben, S., & von Rueden, L. (2021) · Lancaster EPrints (Lancaster University) · doi:10.48550/arxiv.2105.10172 · Cited by 52
“Generating counterfactuals is one direction of providing example-based explanations [74].”
FCE: Feedback Based Counterfactual Explanations for Explainable AI
Suffian, M., Graziani, P., Alonso, J. M., & Bogliolo, A. (2022) · IEEE Access · doi:10.1109/access.2022.3189432 · Cited by 34
Self-citationExplainable AI: To Reveal the Logic of Black-Box Models
Chinu, & Bansal, U. (2023) · New Generation Computing · doi:10.1007/s00354-022-00201-2 · Cited by 33
“ILIA STEPIN et al. [43] classified evaluation metrics into three categories: None, Hybrid, Subjective and Objective Metrics.”
Mathematical optimization modelling for group counterfactual explanations
Carrizosa, E., Ramírez-Ayerbe, J., & Morales, D. R. (2024) · European Journal of Operational Research · doi:10.1016/j.ejor.2024.01.002 · Cited by 30
Pradeep, P., Caro-Martínez, M., & Wijekoon, A. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124733 · Cited by 28
A comprehensive review of explainable AI in cybersecurity: Decoding the black box
Sharma, A., Rani, S., & Shabaz, M. (2025) · ICT Express · doi:10.1016/j.icte.2025.10.004 · Cited by 46
Mariotti, E., Alonso, J. M., & Albert Gatt, A. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101882 · Cited by 28
Self-citationProlog-based agnostic explanation module for structured pattern classification
Nápoles, G., Hoitsma, F., Knoben, A., Jastrzębska, A., & León, M. (2022) · Information Sciences · doi:10.1016/j.ins.2022.12.012 · Cited by 28
Chen, Z., Silvestri, F., Tolomei, G., Wang, J., Zhu, H., & Ahn, H. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3223892 · Cited by 27
Gajcin, J., & Dusparić, I. (2024) · ACM Computing Surveys · doi:10.1145/3648472 · Cited by 36
“In Section 3 we identify the key similarities and differences between supervised and reinforcement learning from the perspective of explain-ability.”
BackgroundDesigning meaningful human oversight in AI
Zhu, L., Lu, Q., Ding, M., Lee, S. U., & Wang, C. (2026) · AI and Ethics · doi:10.1007/s43681-026-01147-7 · Cited by 49
“Much of the XAI literature focuses on explaining model internals or local input–output sensitivities, including saliency maps, feature attributions, and counterfactual explanations [19, 20].”
Evaluating the effectiveness of XAI techniques for encoder-based language models
Mersha, M. A., Yigezu, M. G., & Kalita, J. K. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113042 · Cited by 29
Exploring the Role of Local and Global Explanations in Recommender Systems
Radensky, M., Downey, D., Lo, K., Popović, Z., & Weld, D. S. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3519795 · Cited by 29
“Theymay be factual, explaining why a certain model outcome occurred, or contrastive, explaining why another outcome did not occur; they may also be counterfactual, explaining how another outcome could have occurred instead [18, 22, 51].”
BackgroundCounterfactual explanations as interventions in latent space
Crupi, R., Castelnovo, A., Regoli, D., & González, B. S. M. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00889-2 · Cited by 23
“Since the first proposal of counterfactual explanations by Wachter et al. (2017), a large body of research concerning different algorithms and techniques to generate contrastive and counterfactual explanations have been conducted (Verma et al., 2020; Stepin et al., 2021).”
InfluentialCounterfactual explanations for misclassified images: How human and machine explanations differ
Delaney, E. D., Pakrashi, A., Greene, D., & Keane, M. T. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103995 · Cited by 28
Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Box
Moreira, C., Chou, Y.-L., Hsieh, C., Ouyang, C., Pereira, J. M., & Jorge, J. A. (2024) · ACM Computing Surveys · doi:10.1145/3672553 · Cited by 34
Counterfactual Learning on Graphs: A Survey
Guo, Z., Wu, Z., Xiao, T., Aggarwal, C. C., Liu, H., & Wang, S. (2025) · Machine Intelligence Research · doi:10.1007/s11633-024-1519-z · Cited by 40
Factual and Counterfactual Explanations in Fuzzy Classification Trees
Fernández, G., Aledo, J. A., Gámez, J. A., & Puerta, J. M. (2022) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2022.3179582 · Cited by 22
“Rather, it is common to generate different types of explanations: some that explain why the instance has been classified into a determined category (factual explanations) and others that explain why it has not been classified into a different category (counterfactual explanations) [12].”
BackgroundA Generalized Explanation Framework for Visualization of Deep Learning Model Predictions
Wang, P., & Vasconcelos, N. M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3241106 · Cited by 21
“Bothaimtoanswerquestions“WhyPand notQ,”althoughsomeliteratureemphasizesthatcounterfactualexplanationsshouldgeneratealternativeexamples,illustratinghowobjectschangeforthe alternativedecision[24],[25].”
Harnessing Prior Knowledge for Explainable Machine Learning: An Overview
Beckh, K., Müller, S., Jakobs, M., Toborek, V., Tan, H., Fischer, R., Welke, P., Houben, S., & von Rueden, L. (2023) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00038 · Cited by 23
“a) Formalized Priors for Explanations: Generating counterfactuals is one direction of providing example-based explanations [53].”
BackgroundFormalising the Robustness of Counterfactual Explanations for Neural Networks
Jiang, J., Leofante, F., Rago, A., & Toni, F. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i12.26740 · Cited by 27
Counterfactual Explanations for Machine Learning: Challenges RevisitedPreprint
Verma, S., Dickerson, J. P., & Hines, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.07756 · Cited by 51
“, counterfactual and contrastive explanations [7, 30, 3, 18, 33] by way of broad literature review.”
BackgroundSDA-Vis: A Visualization System for Student Dropout Analysis Based on Counterfactual Exploration
Garcia-Zanabria, G., Gutierrez-Pachas, D. A., Cámara-Chávez, G., Poco, J., & Gomez-Nieto, E. (2022) · Applied Sciences · doi:10.3390/app12125785 · Cited by 20
Learning Time Series Counterfactuals via Latent Space Representations
Wang, Z., Samsten, I., Mochaourab, R., & Papapetrou, P. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-88942-5_29 · Cited by 21
Mapping the landscape of ethical considerations in explainable AI research
Nannini, L., Manerba, M. M., & Beretta, I. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09773-7 · Cited by 33
“…ethics principles while identifying and reporting various explanation techniques and domain, shedding light on diverse domain applications (Samek et al., 2017; Adadi & Berrada, 2018; Arrieta et al., 2020; Cambria et al., 2023; Stepin et al., 2021; Saeed & Omlin, 2023a; Martins et al., 2024).”
Effective Explanations for Entity Resolution Models
Teofili, T., Firmani, D., Koudas, N., Martello, V., Merialdo, P., & Srivastava, D. (2022) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde53745.2022.00248 · Cited by 22
Glacier: guided locally constrained counterfactual explanations for time series classification
Wang, Z., Samsten, I., Miliou, I., Mochaourab, R., & Papapetrou, P. (2024) · Machine Learning · doi:10.1007/s10994-023-06502-x · Cited by 24
“Interest in counterfactual explanations has surged in the last few years in several high-stake applications (Stepin et al., 2021).”
Explainable artificial intelligence for molecular design in pharmaceutical research
Lamens, A., & Bajorath, J. (2026) · Chemical Science · doi:10.1039/d5sc08461j · Cited by 21
“33,34 Therefore, an expected prediction (P) is de ned as the ‘ fact ’ and an alternative (unexpected) prediction (Q) as the ‘ foil ’ .”
InfluentialOn Explainable AI and Abductive Inference
Medianovskyi, K., & Pietarinen, A. (2022) · Philosophies · doi:10.3390/philosophies7020035 · Cited by 19
“In ML, CF is expected to typify the kinds of generic but example-laden explanations that give the explainees what they seem to be seeking in explanations: small variations in the input data that would or could merit a reassessment or retraction of the actual proposed action [8,9].”
BackgroundStepin, I., Alonso, J. M., Catalá, A., & Pereira-Fariña, M. (2022) · Information Sciences · doi:10.1016/j.ins.2022.10.098 · Cited by 22
Self-citationEldrandaly, K. A., Abdel‐Basset, M., Ibrahim, M., & AbdelAziz, N. M. (2022) · Enterprise Information Systems · doi:10.1080/17517575.2022.2098537 · Cited by 19
“Where (Chou et al. 2022) and (Stepin et al. 2021) present a systematic literature review on counterfactuals and causability for XAI.”
BackgroundIs the transposition of EU directives homogeneous? An AI counterfactual example
Ballestar, M. T., Sáinz, J., & Sanz, I. (2021) · Applied Economics Letters · doi:10.1080/13504851.2021.1967860 · Cited by 17
“Moreover, some countries can change their classification over time (Stepin et al. 2021).”
BackgroundZhang, M., Pan, Y., Cui, Q., Lü, Y., & Yu, W. (2025) · Experimental Gerontology · doi:10.1016/j.exger.2025.112812 · Cited by 20
Bauer, J. M., & Michalowski, M. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf110 · Cited by 19
An Overview of Explainable AI Methods, Forms and Frameworks
Kumar, D., & Mehta, M. A. (2022) · Intelligent systems reference library · doi:10.1007/978-3-031-12807-3_3 · Cited by 19
An interdisciplinary perspective on AI-supported decision making in medicine
Ammeling, J., Aubreville, M., Fritz, A., Kießig, A., Krügel, S., & Uhl, M. (2024) · Technology in Society · doi:10.1016/j.techsoc.2024.102791 · Cited by 20
From Explanation to Recommendation: Ethical Standards for Algorithmic Recourse
Sullivan, E., & Verreault-Julien, P. (2022) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3514094.3534185 · Cited by 18
“Counterfactual generation serves as an explanation method because finding the smallest changes that would flip a decision tells us important information regarding how a model made its decision [48].”
BackgroundBlack-box error diagnosis in Deep Neural Networks for computer vision: a survey of tools
Fraternali, P., Milani, F., Torres, R. N., & Zangrando, N. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-08100-9 · Cited by 17
“• Counterfactual explanations: is the approach that exposes the model behavior by showing how some actions, such as a change to the input, alter the behavior of the system [17, 18].”
BackgroundConfalonieri, R., & Alonso, J. M. (2023) · IEEE Intelligent Systems · doi:10.1109/mis.2023.3334639 · Cited by 16
“13,14 More recently, Ali et al. 1 provided a review of XAI techniques from four axes, namely, data explainability, model explainability, posthoc explainability, and assessment of explanations.”
BackgroundSelf-citationCounterfactual simulation in causal cognition
Gerstenberg, T. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.04.012 · Cited by 26
A Review of Interpretability Methods for Gradient Boosting Decision Trees
Gonçalves, V. S. F., & de Carvalho, V. R. (2025) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2025.5324 · Cited by 16
“Furthermore, 6 surveys examine interpretability concerning specific machine learning methods or explainability techniques ([Stepin et al ., 2021] [de Oliveira and Martens, 2021] [Covert et al ., 2022] [Sa-hakyan et al ., 2021] [Burkart and Huber, 2021] [Yasodhara et al ., 2021]) examine…”
Explaining black-box classifiers: Properties and functions
Leila Amgoud, A. (2023) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2023.01.004 · Cited by 15
Oliveira, R., Sörensen, K., & Martens, D. (2023) · European Journal of Operational Research · doi:10.1016/j.ejor.2023.08.031 · Cited by 16
Diffeomorphic Counterfactuals With Generative Models
Dombrowski, A.-K., Gerken, J. E., Müller, K., & Kessel, P. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3339980 · Cited by 26
“Adifferentapproach to explain a neural network is given by providing counterfactuals to the original inputs [41], [42], [43].”
BackgroundHow to Build Self-Explaining Fuzzy Systems: From Interpretability to Explainability [AI-eXplained]
Stepin, I., Suffian, M., Catalá, A., & Alonso, J. M. (2024) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2023.3328098 · Cited by 15
“Such so-called counterfactual explanations presup-pose minimal changes in thefeature values of the test instance for the prediction to change, making it maximally relevant to the test instance under consideration [5].”
BackgroundSelf-citationSuffian, M., Kuhl, U., Bogliolo, A., & Alonso-Moral, J. M. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103484 · Cited by 18
Lamens, A., & Bajorath, J. (2023) · ChemMedChem · doi:10.1002/cmdc.202300586 · Cited by 13
Metadata
- Authors
- Stepin, I., Alonso, J. M., Catala, A., & Pereira-Farina, M.
- Affiliations
- Universidade de Santiago de Compostela (ES)
- Venue
- IEEE Access
- Published
- 2021
- Volume
- 9
- Pages
- 11974-12001
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3125997628
- Semantic Scholar
- e016ee7bfc73cb5b8a92f6c517389be837c035eb
- Cited by
- 376
- References
- 176
- Citations by year
- 2026: 47 · 2025: 80 · 2024: 75 · 2023: 79 · 2022: 55 · 2021: 39
- FWCI
- 21.91
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- IEEE Access (publishedVersion, cc-by-nc-nd)
- Repositorio institucional da Universidade de Santiago de Compostela (University of Santiago de Compostela) (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Hispana (submittedVersion, cc-by-nc-nd)
- Funders
- Spanish Ministry of Science, Innovation and Universities (RTI2018-099646-B-I00, RED2018-102641-T)
- Galician Ministry of Education, University and Professional Training (ED431F 2018/02, ED431C 2018/29, ED431G/08, ED431G2019/04)
- European Regional Development Fund
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- contrastive explanations, counterfactual explanations, explanation generation, explainable AI, evidence-based explanations, model interpretability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
