Factual and Counterfactual Explanations for Black Box Decision Making
Open accessThe DOI matches its Crossref record; 12 of the 14 references it cites were found in a registry or an index.
Abstract
The rise of sophisticated machine learning models has brought accurate but obscure decision systems, which hide their logic, thus undermining transparency, trust, and the adoption of artificial intelligence (AI) in socially sensitive and safety-critical contexts. We introduce a local rule-based explanation method, providing faithful explanations of the decision made by a black box classifier on a specific instance. The proposed method first learns an interpretable, local classifier on a synthetic neighborhood of the instance under investigation, generated by a genetic algorithm. Then, it derives from the interpretable classifier an explanation consisting of a decision rule, explaining the factual reasons of the decision, and a set of counterfactuals, suggesting the changes in the instance features that would lead to a different outcome. Experimental results show that the proposed method outperforms existing approaches in terms of the quality of the explanations and of the accuracy in mimicking the black box.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Anchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 1×
“We also evaluate the rule coverage, namely the fraction of instances to explain covered by the rule: large values of coverage means better rule generalization.(4) Figure 8 also shows the average coverage of the decision rules, with LORE showing a consistently better coverage than ANCHOR.”
Counterfactual explanations without opening the black box: automated decisions and the GDPR
Wachter, S., Mittelstadt, B., & Russell, C. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3063289 · Cited by 1,061
Evolutionary Computation 1: Basic Algorithms and Operators
(2018) · CRC Press eBooks · doi:10.1201/9781482268713 · Cited by 634
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629
Self-citationEvolutionary Computation 2: Advanced Algorithms and Operators
(2000) · Taylor & Francis eBooks · doi:10.1201/9781420034349 · Cited by 451
A review of instance selection methods
Olvera-López, J. A., Carrasco-Ochoa, J. A., Martínez-Trinidad, J. F., & Kittler, J. (2010) · Artificial Intelligence Review · doi:10.1007/s10462-010-9165-y · Cited by 394 · cited 1×
“This task differs from approaches to instance selection based on genetic algorithms.(7) In our case, we cannot assume the availability of the training set of b, or not even that b is a supervised ML predictor for which a training set exists.”
MethodCounterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning
Byrne, R. M. J. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/876 · Cited by 285 · cited 1×
“the cognitive psychology literature maintaining that counterfactuals help people to reason on explanations that identify cause-effect or reason-action relations between events.(5,6) Similarly, Grover et al.”
BackgroundA Survey on Evolutionary Instance Selection and Generation
Derrac, J., García, S., & Herrera, F. (2010) · International Journal of Applied Metaheuristic Computing · doi:10.4018/jamc.2010102604 · Cited by 75 · cited 1×
“Our task is instead similar to instance generation in the field of active learning, including evolutionary approaches.(8) We adopt an approach based on a genetic algorithm, which generates z 2 Z1⁄4 [ Z61⁄4 by maximizing the following fitness functions:”
Method
Cited by
Díaz-Rodríguez, N., Del Ser, J., Coeckelbergh, M., López de Prado, M., Herrera-Viedma, E., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101896 · Cited by 800
Counterfactual explanations and how to find them: literature review and benchmarking
Guidotti, R. (2024) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00831-6 · Cited by 471
Self-citationStepin, I., Alonso, J. M., Catala, A., & Pereira-Farina, M. (2021) · IEEE Access · doi:10.1109/access.2021.3051315 · Cited by 376
“In contrast to the model-specific frameworks operating on decision tree output, Guidotti et al. employ decision trees as part of reconstructing the reasoning behind any arbitrary classifier in a post-hoc fashion [140].”
BackgroundResultInfluentialDeterministic Local Interpretable Model-Agnostic Explanations for Stable Explainability
Zafar, M. R., & Khan, N. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3030027 · Cited by 374
“In the same vein, Leave-One-Covariate-Out (LOCO) [12] and Local Rule-based Explanation (LORE) [13] are other popular techniques for generating local explanation models that offer local variable importance measures.”
Islam, M. R., Ahmed, M. U., Barua, S., & Begum, S. (2022) · Applied Sciences · doi:10.3390/app12031353 · Cited by 311
Metadata
- Authors
- Guidotti, R., Monreale, A., Giannotti, F., Pedreschi, D., Ruggieri, S., & Turini, F.
- Affiliations
- Istituto di Scienza e Tecnologie dell'Informazione "Alessandro Faedo" (IT)
- University of Pisa (IT)
- ORCID
- Venue
- IEEE Intelligent Systems
- Published
- 2019-11-01
- Volume
- 34
- Issue
- 6
- Pages
- 14-23
- ISSN
- 1541-1672 (ISSN-L)
- 1941-1294
- OpenAlex
- W2994120362
- Cited by
- 349
- References
- 14
- Citations by year
- 2026: 38 · 2025: 67 · 2024: 62 · 2023: 60 · 2022: 47 · 2021: 48 · 2020: 26 · 2019: 1
- FWCI
- 13.92
- Type
- journal article
- Language
- en
- Open access
- green
- ieeexplore.ieee.org
- Licence
- Locations
- IEEE Intelligent Systems (publishedVersion)
- CINECA IRIS Institutial research information system (University of Pisa) (submittedVersion, other-oa)
- ISTI Open Portal (submittedVersion, other-oa)
- Scuola Normale Superiore di Pisa (submittedVersion)
- Funders
- European Commission (INFRAIA-1-2014-2015)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Data Mining Algorithms and Applications
- Keywords
- explainable AI, black-box classifiers, counterfactual explanations, genetic algorithm, interpretable classifiers, model transparency, trust in AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Factual and Counterfactual Explanations for Black Box Decision Making. https://app.cochat.ai/papers/10.1109/mis.2019.2957223
- The paper
- Guidotti, R., Monreale, A., Giannotti, F., Pedreschi, D., Ruggieri, S., & Turini, F. (2019). Factual and Counterfactual Explanations for Black Box Decision Making. IEEE Intelligent Systems. https://doi.org/10.1109/mis.2019.2957223
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.