Factual and Counterfactual Explanations for Black Box Decision Making
Open accessThe DOI matches its Crossref record; 12 of the 14 references it cites were found in a registry or an index.
Abstract
The rise of sophisticated machine learning models has brought accurate but obscure decision systems, which hide their logic, thus undermining transparency, trust, and the adoption of artificial intelligence (AI) in socially sensitive and safety-critical contexts. We introduce a local rule-based explanation method, providing faithful explanations of the decision made by a black box classifier on a specific instance. The proposed method first learns an interpretable, local classifier on a synthetic neighborhood of the instance under investigation, generated by a genetic algorithm. Then, it derives from the interpretable classifier an explanation consisting of a decision rule, explaining the factual reasons of the decision, and a set of counterfactuals, suggesting the changes in the instance features that would lead to a different outcome. Experimental results show that the proposed method outperforms existing approaches in terms of the quality of the explanations and of the accuracy in mimicking the black box.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Anchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 1×
“We also evaluate the rule coverage, namely the fraction of instances to explain covered by the rule: large values of coverage means better rule generalization.(4) Figure 8 also shows the average coverage of the decision rules, with LORE showing a consistently better coverage than ANCHOR.”
Counterfactual explanations without opening the black box: automated decisions and the GDPR
Wachter, S., Mittelstadt, B., & Russell, C. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3063289 · Cited by 1,061
Evolutionary Computation 1: Basic Algorithms and Operators
(2018) · CRC Press eBooks · doi:10.1201/9781482268713 · Cited by 634
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629
Self-citationEvolutionary Computation 2: Advanced Algorithms and Operators
(2000) · Taylor & Francis eBooks · doi:10.1201/9781420034349 · Cited by 451
A review of instance selection methods
Olvera-López, J. A., Carrasco-Ochoa, J. A., Martínez-Trinidad, J. F., & Kittler, J. (2010) · Artificial Intelligence Review · doi:10.1007/s10462-010-9165-y · Cited by 394 · cited 1×
“This task differs from approaches to instance selection based on genetic algorithms.(7) In our case, we cannot assume the availability of the training set of b, or not even that b is a supervised ML predictor for which a training set exists.”
MethodCounterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning
Byrne, R. M. J. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/876 · Cited by 285 · cited 1×
“the cognitive psychology literature maintaining that counterfactuals help people to reason on explanations that identify cause-effect or reason-action relations between events.(5,6) Similarly, Grover et al.”
BackgroundA Survey on Evolutionary Instance Selection and Generation
Derrac, J., García, S., & Herrera, F. (2010) · International Journal of Applied Metaheuristic Computing · doi:10.4018/jamc.2010102604 · Cited by 75 · cited 1×
“Our task is instead similar to instance generation in the field of active learning, including evolutionary approaches.(8) We adopt an approach based on a genetic algorithm, which generates z 2 Z1⁄4 [ Z61⁄4 by maximizing the following fitness functions:”
Method
Cited by
Items 401–438 of 438
From Black Boxes to Clear Boxes: A Human-Centered Review of Explainable AI Methods
Sanjana, A., & Nagaraj, S. (2026) · International Conference on Inventive Computation Technologies (ICICT) · doi:10.1109/icict68280.2026.11510770
“[19].”
Interpreting Federated Learning by Aggregating SHAP Explanations
Bonsignori, V., Corbucci, L., Naretto, F., Guidotti, R., & Monreale, A. (2026) · IEEE Access · doi:10.1109/access.2026.3696009
“The growing interest in the XAI research field [6], [26], [40] led to the development of multiple techniques designed to explain models trained in a centralised scenario [28], [29], [41], [50].”
Self-citationExplainability of Large Language Models in Healthcare Information Systems: A Review
Sullivan, R., & Elsayed, N. (2026) · IEEE International Conference on Computing and Machine Intelligence (ICMI) · doi:10.1109/icmi68585.2026.11539937
“The counterfactual concept manifested itself in another study that highlighted the applicability of local explanation in healthcare [25] that used the Local Rule-Based Explanations (LORE) technique [17].”
Symb-xMIL: Symbolic Explanations for Multiple Instance Learning in Digital PathologyPreprint
Luo, Y., Hense, J., Prenißl, N., Möck, A., Müller, K.-R., Schnake, T., & Idaji, M. J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.06224
“To address these limitations, we augment heatmap-based explanations with a structured, symbolic representation of model reasoning, enabling both local and global interpretability [2, 19, 44].”
When the Group Agrees: Explaining Collective Recommendations From Core to Consensus
Yera, R., & Leão, M. T. D. S. (2026) · Expert Systems · doi:10.1111/exsy.70317
“While we employed a traditional approach (i.e., FP-Growth) to identify the item-item relationships, we here obtain the feature-item relationships through a novel approach, inspired by LORE (Guidotti et al. 2019), a popular local rule-based explanation method for tabular data.”
InfluentialAraújo, G., de Sena, P. A. P., & Oliveira, M. D. L. (2026) · doi:10.62140/gapsmo990233160
Explicando os “E se?” por Trás da Detecção de Violência Física Infantil: Uma Abordagem Contrafactua
Xavier, L., Balbino, M., Mannan, H., & Nobre, C. (2026) · doi:10.5753/sbcas.2026.21600
On the Interplay of Explainability and Fairness in AI: A Survey
Fragkathoulas, C., Papanikou, V., Karidi, D. P., & Pitoura, E. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3706635
“Approaches within this category include surrogate based [5], [51] explanations, which fit linear models near the explained datapoint, and rule based ones [51], which extract rules from these models [29].”
Quantifying and Mitigating Epistemic Uncertainty in Local Rule-Based Explanations
Bobek, S., & Mozolewski, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29918-5_29
“…[12] ICCS Camera Ready Version 2026 To cite this paper please use the final published version: DOI: 10.1007/978-3-032-29918-5_29 and rule-based ones [14, 21, 8, 18, 23], empirical studies indicate that experts and users strongly prefer those that offer structural clarity, explicit decision logic,…”
InfluentialHITLAP: Human in the Loop AI-Privacy Architecture for Next Generation Banking Automation
Koganti, H., Refonaa, J., Ignatius, A. G., Rao, A. V. S. S. R., Praba, R. S., & Keerthana, N. V. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8217-3.ch006
Bayrak, B., Güllü, M., Karahan, S. N., & ÇELİK, B. D. (2026) · IEEE Access · doi:10.1109/access.2026.3708897
“LORE: LORE [39] explains a black-box prediction by fitting a local surrogate decision tree on a synthetically generated neighbourhood around the query instance, then deriving a counterfactual by minimally perturbing features along the tree’s decision path.”
Immersive 3D Interface for Explainable Movie Recommendations
Ortega, O., Labella, Á., Yera, R., & Martínez, L. M. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2497-5_31
Francescomarino, C. D., Ghidini, C., Maggi, F. M., Oukharijane, J., & Rizzi, W. (2026) · Data & Knowledge Engineering · doi:10.1016/j.datak.2026.102625
Zhao, M., Yang, G., Zhang, Z., Biljecki, F., Zu, H., & Wu, C. (2026) · Building and Environment · doi:10.1016/j.buildenv.2026.115095
Sinlae, A. A. J., Fajriana, F., Suzana, ., & Arianty, K. P. (2026) · International Journal of Engineering Science and Information Technology · doi:10.52088/ijesty.v6i3.1882
“Although many machine learning algorithms achieve high predictive accuracy, particularly complex models, their internal decision-making processes are often difficult to interpret, leading them to be characterized as black-box models [21,22].”
A Survey of Causal Learning in Graph-Based Recommender Systems
Yu, D., Li, Q., Soundararaj, B., Pettit, C. J., & Xu, G. (2026) · ACM Computing Surveys · doi:10.1145/3840748
Kawazura, Y., & Saitou, D. (2026) · IEEE Access · doi:10.1109/access.2026.3727143
“[4], [12], [13].”
Explainable AI: Building Trustworthy Deep Learning Systems
Mala, D. J., & Ganesan, S. (2026) · CRC Press eBooks · doi:10.1201/9781003630241
Evaluating the Robustness of Counterfactual Generating Methods for Tree-Based Ensembles
Wijethunga, R., Galwaduge, V., Sadhu, A., & Samarabandu, J. K. (2026) · Canadian Journal of Electrical and Computer Engineering · doi:10.1109/icjece.2026.3685607
“Although the aforementioned CF methods can be applied to these glass-box models, CFs can also be extracted trivially from themselves [9].”
Проблеми інформатизації. Том 3
(2019) · eNTUKhPIIR Repository (Kharkiv Polytechnic Institute)
Mozolewski, M., Bobek, S., & Nalepa, G. J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7422878
Cetinkaya, N. E., & Krämer, N. C. (2026) · Journal of Psychology and AI · doi:10.1080/29974100.2026.2758023
Rahmati, M., & Rahmati, N. (2026) · Journal of Electrical Systems and Information Technology · doi:10.1186/s43067-026-00401-6
“Security-specific explainability research has developed techniques for interpreting intrusion detection systems, malware classifiers, and vulnerability prediction models [63].”
Expert-Guided Counterfactual Explanations for Credit Risk
Benassi, R., Paganelli, M., Aloe, G., Bagli, E., Carducci, M. C., Giuffrida, P., Lavecchia, N., Squadrani, M., Oliva, E. M. R., Vincini, M., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39820-8_25
Zia, R., & Malik, M. S. I. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.28565
“Counterfactual methods take a different route entirely, explaining a prediction by showing the minimal input change that would have produced a different outcome [11].”
A Synthetic Ground-Truth Framework for the Evaluation of Explainable AI MethodsPreprint
Miró-Nicolau, M., Spinnato, F., & Guidotti, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.30397
“In total, we evaluate nine approaches: RISE [42], LIME [43], Gradient [48], DeepLIFT [47], KernelSHAP [31], Integrated Gradients (IG) [51], T-SHAP [40] (in both ROI and Window variants), and LORE [15].”
InfluentialSelf-citationChakraborty, D., Adnan, A., & Dey, E. K. (2026) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2026.3738310
“Counterfactual explanation is an XAI technique that can extend interpretability by identifying how minimal modifications in geometric features can alter the predicted class ofapoint[33].”
Bisection-Based Neighborhood Generation for Local Rule-Based Explainable Artificial Intelligence
Baldo, S., Zhao, L., & Tinós, R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39882-6_40
Ali, A. (2026) · Annual Methodological Archive Research Review
Pareto Optimal Counterfactual Explanations with an LLM Validation
Lim, S., Prade, H. M., & Richard, G. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-6513-8_27
How Graphs Ground Large Language Models Counterfactuals for Subgraph Verbalizations
Buchmann, S., Slany, E., & Scheele, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-42015-2_23
Artificial Intelligence and Inventive Step: A European Patent Law Perspective
Chiettini, A. (2026) · Munich studies on innovation and competition · doi:10.1007/978-3-032-41497-7
Attention to those tokens! Ensemble Clustering can explain Attention-based Text Classifiers
Angiulli, F., Luca, F. D., & Fassetti, F. (2026) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3799682.3839930
Dubois, P., Lefevre, M. A., Moreau, C., Martin, J., & Dupont, T. (2025) · Journal of Trustworthy and Explainable Intelligent Systems · doi:10.68159/r221092846
Kumar, A., & Das, T. K. (2026) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3858915
Supporting information 2 of Matrix Profile-Based Interpretable Time Series Classifier
Guidotti, R., & D’Onofrio, M. (2021) · doi:10.3389/frai.2021.699448.s002
Self-citation
Metadata
- Authors
- Guidotti, R., Monreale, A., Giannotti, F., Pedreschi, D., Ruggieri, S., & Turini, F.
- Affiliations
- Istituto di Scienza e Tecnologie dell'Informazione "Alessandro Faedo" (IT)
- University of Pisa (IT)
- ORCID
- Venue
- IEEE Intelligent Systems
- Published
- 2019-11-01
- Volume
- 34
- Issue
- 6
- Pages
- 14-23
- ISSN
- 1541-1672 (ISSN-L)
- 1941-1294
- OpenAlex
- W2994120362
- Cited by
- 349
- References
- 14
- Citations by year
- 2026: 38 · 2025: 67 · 2024: 62 · 2023: 60 · 2022: 47 · 2021: 48 · 2020: 26 · 2019: 1
- FWCI
- 13.92
- Type
- journal article
- Language
- en
- Open access
- green
- ieeexplore.ieee.org
- Licence
- Locations
- IEEE Intelligent Systems (publishedVersion)
- CINECA IRIS Institutial research information system (University of Pisa) (submittedVersion, other-oa)
- ISTI Open Portal (submittedVersion, other-oa)
- Scuola Normale Superiore di Pisa (submittedVersion)
- Funders
- European Commission (INFRAIA-1-2014-2015)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Data Mining Algorithms and Applications
- Keywords
- explainable AI, black-box classifiers, counterfactual explanations, genetic algorithm, interpretable classifiers, model transparency, trust in AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.