Factual and Counterfactual Explanations for Black Box Decision Making
Open accessThe DOI matches its Crossref record; 12 of the 14 references it cites were found in a registry or an index.
Abstract
The rise of sophisticated machine learning models has brought accurate but obscure decision systems, which hide their logic, thus undermining transparency, trust, and the adoption of artificial intelligence (AI) in socially sensitive and safety-critical contexts. We introduce a local rule-based explanation method, providing faithful explanations of the decision made by a black box classifier on a specific instance. The proposed method first learns an interpretable, local classifier on a synthetic neighborhood of the instance under investigation, generated by a genetic algorithm. Then, it derives from the interpretable classifier an explanation consisting of a decision rule, explaining the factual reasons of the decision, and a set of counterfactuals, suggesting the changes in the instance features that would lead to a different outcome. Experimental results show that the proposed method outperforms existing approaches in terms of the quality of the explanations and of the accuracy in mimicking the black box.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Anchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 1×
“We also evaluate the rule coverage, namely the fraction of instances to explain covered by the rule: large values of coverage means better rule generalization.(4) Figure 8 also shows the average coverage of the decision rules, with LORE showing a consistently better coverage than ANCHOR.”
Counterfactual explanations without opening the black box: automated decisions and the GDPR
Wachter, S., Mittelstadt, B., & Russell, C. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3063289 · Cited by 1,061
Evolutionary Computation 1: Basic Algorithms and Operators
(2018) · CRC Press eBooks · doi:10.1201/9781482268713 · Cited by 634
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629
Self-citationEvolutionary Computation 2: Advanced Algorithms and Operators
(2000) · Taylor & Francis eBooks · doi:10.1201/9781420034349 · Cited by 451
A review of instance selection methods
Olvera-López, J. A., Carrasco-Ochoa, J. A., Martínez-Trinidad, J. F., & Kittler, J. (2010) · Artificial Intelligence Review · doi:10.1007/s10462-010-9165-y · Cited by 394 · cited 1×
“This task differs from approaches to instance selection based on genetic algorithms.(7) In our case, we cannot assume the availability of the training set of b, or not even that b is a supervised ML predictor for which a training set exists.”
MethodCounterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning
Byrne, R. M. J. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/876 · Cited by 285 · cited 1×
“the cognitive psychology literature maintaining that counterfactuals help people to reason on explanations that identify cause-effect or reason-action relations between events.(5,6) Similarly, Grover et al.”
BackgroundA Survey on Evolutionary Instance Selection and Generation
Derrac, J., García, S., & Herrera, F. (2010) · International Journal of Applied Metaheuristic Computing · doi:10.4018/jamc.2010102604 · Cited by 75 · cited 1×
“Our task is instead similar to instance generation in the field of active learning, including evolutionary approaches.(8) We adopt an approach based on a genetic algorithm, which generates z 2 Z1⁄4 [ Z61⁄4 by maximizing the following fitness functions:”
Method
Cited by
Díaz-Rodríguez, N., Del Ser, J., Coeckelbergh, M., López de Prado, M., Herrera-Viedma, E., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101896 · Cited by 800
Counterfactual explanations and how to find them: literature review and benchmarking
Guidotti, R. (2024) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00831-6 · Cited by 471
Self-citationStepin, I., Alonso, J. M., Catala, A., & Pereira-Farina, M. (2021) · IEEE Access · doi:10.1109/access.2021.3051315 · Cited by 376
“In contrast to the model-specific frameworks operating on decision tree output, Guidotti et al. employ decision trees as part of reconstructing the reasoning behind any arbitrary classifier in a post-hoc fashion [140].”
BackgroundResultInfluentialDeterministic Local Interpretable Model-Agnostic Explanations for Stable Explainability
Zafar, M. R., & Khan, N. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3030027 · Cited by 374
“In the same vein, Leave-One-Covariate-Out (LOCO) [12] and Local Rule-based Explanation (LORE) [13] are other popular techniques for generating local explanation models that offer local variable importance measures.”
Islam, M. R., Ahmed, M. U., Barua, S., & Begum, S. (2022) · Applied Sciences · doi:10.3390/app12031353 · Cited by 311
Bias and Discrimination in AI: A Cross-Disciplinary Perspective
Ferrer, X., Nuenen, T. V., Such, J. M., Cote, M., & Criado, N. (2021) · IEEE Technology and Society Magazine · doi:10.1109/mts.2021.3056293 · Cited by 308
“, by looking at statistical parity), implementations can compare the algorithmic outcomes obtained by two different subpopulations in the data set [13], or make use of counterfactual or contrastive explanations [8], [11], [14], which have shown promising results in aiding the provision of interpretable models and make…”
BackgroundBenchmarking and survey of explanation methods for black box models
Bodria, F., Giannotti, F., Guidotti, R., Naretto, F., Pedreschi, D., & Rinzivillo, S. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-023-00933-9 · Cited by 264
Self-citationKeane, M. T., & Smyth, B. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58342-2_11 · Cited by 128
Classification of Explainable Artificial Intelligence Methods through Their Output Formats
Vilone, G., & Longo, L. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3030032 · Cited by 152
“A genetic algorithm is also employed in GLocalX [71,72] to generate local rules that explain the prediction made by a classifier on a specific instance.”
BackgroundMethodInterpretability research of deep learning: A literature survey
Xu, B., & Yang, G. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102721 · Cited by 214
Wani, N. A., Kumar, R., Mamta, Bedi, J., & Rida, I. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102472 · Cited by 191
Reviewing the Need for Explainable Artificial Intelligence (xAI)
Gerlings, J., Shollo, A., & Constantiou, I. (2021) · Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.24251/hicss.2021.156 · Cited by 115
Instance-Based Counterfactual Explanations for Time Series Classification
Delaney, E. D., Greene, D., & Keane, M. T. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86957-1_3 · Cited by 188
Explainable Predictive Maintenance of Rotating Machines Using LIME, SHAP, PDP, ICE
Gawde, S., Patil, S., Kumar, S., Kamat, P. V., Kotecha, K. V., & Alfarhood, S. (2024) · IEEE Access · doi:10.1109/access.2024.3367110 · Cited by 185
Explainable AI for Medical Data: Current Methods, Limitations, and Future Directions
Hossain, M. I., Zamzmi, G., Mouton, P. R., Salekin, M. S., Sun, Y., & Goldgof, D. B. (2025) · ACM Computing Surveys · doi:10.1145/3637487 · Cited by 184
“Hence, current XAI methods should be examined for complex architectures and multimodal datasets.”
XCM: An Explainable Convolutional Neural Network for Multivariate Time Series Classification
Fauvel, K., Lin, T., Masson, V., Fromont, É., & Termier, A. (2021) · Mathematics · doi:10.3390/math9233137 · Cited by 177
“, LIME [31], SHAP [32], Anchors [33] and LORE [34]).”
BackgroundMethodHatt, M., Krizsan, A. K., Rahmim, A., Bradshaw, T., Costa, P. F., Forgács, A., Seifert, R., Zwanenburg, A., Naqa, I. E., Kinahan, P. E., et al. (2022) · European Journal of Nuclear Medicine and Molecular Imaging · doi:10.1007/s00259-022-06001-6 · Cited by 176
Degas, A., Islam, M. R., Hurter, C., Barua, S., Rahman, H., Poudel, M., Ruscio, D., Ahmed, M. U., Begum, S., Rahman, M. A., et al. (2022) · Applied Sciences · doi:10.3390/app12031295 · Cited by 168
How to certify machine learning based safety-critical systems? A systematic literature review
Tambon, F., Laberge, G., An, L., Nikanjam, A., Mindom, P. S. N., Pequignot, Y., Khomh, F., Antoniol, G., Merlo, E., & Laviolette, F. (2022) · Automated Software Engineering · doi:10.1007/s10515-022-00337-x · Cited by 157
“– Explanations can take several forms : feature importance [181, 224], counterfactual examples [90, 166], textual description [6], and model visualisation [106, 152] – The two main paradigms of eXplainable Artificial Intelligence are the training of intrinsically interpretable yet accurate models, and the use of post-…”
BackgroundMethodKhan, F. S., Mazhar, S. S., Mazhar, K., AlSaleh, D., & Mazhar, A. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11215-9 · Cited by 151
GLocalX—From Local to Global Explanations of Black Box AI Models
Setzu, M., Guidotti, R., Monreale, A., Turini, F., Pedreschi, D., & Giannotti, F. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103457 · Cited by 140
“For the local rule extraction we selected lore [23] due to its high fidelity and stability [25].”
MethodSelf-citationEvaluating local explanation methods on ground truth
Guidotti, R. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2020.103428 · Cited by 98
Self-citationShaban, O. S., & Omoush, A. (2025) · Sustainability · doi:10.3390/su17093818 · Cited by 119
Riveiro, M., & Thill, S. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103507 · Cited by 79
“Table 2 Measures and example questions used for each”
MethodMartins, T., de Almeida, A., Cardoso, E., & Nunes, L. (2023) · IEEE Access · doi:10.1109/access.2023.3347028 · Cited by 98
“The first dataset, 10 used in [59], [61], aims to predict whether a person makes over $50,000 a year, containing 14 features and 48,842 records.”
BackgroundMethodPrado-Romero, M. A., Prenkaj, B., Stilo, G., & Giannotti, F. (2024) · ACM Computing Surveys · doi:10.1145/3618105 · Cited by 92
“While white-box models are preferred when making critical decisions that might change the course of events in a business [109], black-box models are more performant and can generalise even with high-dimensional input features in scenarios such as predicting protein relations [64], student dropout…”
BackgroundSelf-citationArtificial Intelligence in Hypertension: Seeing Through a Glass Darkly
Padmanabhan, S., Tran, T. Q. B., & Dominiczak, A. F. (2021) · Circulation Research · doi:10.1161/circresaha.121.318106 · Cited by 78
“Other strategies include counterfactual explanations 89 or using or utilizing a model with multistage architecture, allowing a clinician to inspect and gain potential insight into the model’s decision process.”
Metta, C., Beretta, A., Pellungrini, R., Rinzivillo, S., & Giannotti, F. (2024) · Bioengineering · doi:10.3390/bioengineering11040369 · Cited by 87
“This predictor, in turn, facilitates the generation of meaningful explanations that include decision rules and coun-terfactual scenarios, thereby illuminating the influence of specific factors on outcomes [1].”
BackgroundMethodInfluentialSelf-citationXEM: An explainable-by-design ensemble method for multivariate time series classification
Fauvel, K., Fromont, É., Masson, V., Faverdin, P., & Termier, A. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00823-6 · Cited by 85
“There are two methods adopting an implicit approach: Mixture of Experts (ME) (Jacobs et al. 1991) and Negative Correlation Learning (NCL) (Liu and Yao 1999).”
MethodBeyond Trivial Counterfactual Explanations with Diverse Valuable Explanations
Rodríguez, P., Caccia, M., Lacoste, A., Zamparo, L., Laradji, I., Charlin, L., & Vázquez, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00109 · Cited by 81
“[14] first construct a local neighbourhood around test instances, finding both positive and negative exemplars within the neighbourhood.”
BackgroundMaking AI Explainable in the Global South: A Systematic Review
Okolo, C. T., Dell, N. L., & Vashistha, A. (2022) · ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies (COMPASS) · doi:10.1145/3530190.3534802 · Cited by 81
“Other methods from philosophical domains include contrastive explanations [33, 64] which aim to describe event occurrences in contrast to another and counterfactuals [24, 51, 70] which describe events in causal form.”
Lee, M. H., & Chew, C. J. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610218 · Cited by 78
“The counterfactual explanations describe what changes in feature values lead to updating an AI output in a certain way [12, 24, 47].”
BackgroundMethodN-doped LaPO4: An effective Pt-free catalyst for electrocatalytic oxygen reduction
Zhao, R., Chen, Z., Li, Q., Wang, X., Tang, Y., Fu, G., Li, H., Lee, J., & Huang, S. (2022) · Chem Catalysis · doi:10.1016/j.checat.2022.11.008 · Cited by 78
Fedele, A., Punzi, C., & Tramacere, S. (2024) · Computer law & security review · doi:10.1016/j.clsr.2024.105986 · Cited by 76
Generalization in neural networks: A broad survey
Rohlfs, C. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128701 · Cited by 68
Bueff, A., Cytryński, M., Calabrese, R., Jones, M., Roberts, J., Moore, J., & Brown, I. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117271 · Cited by 64
The Robustness of Counterfactual Explanations Over Time
Ferrario, A., & Loi, M. (2022) · IEEE Access · doi:10.1109/access.2022.3196917 · Cited by 64
On the robustness of sparse counterfactual explanations to adverse perturbations
Virgolin, M., & Fracaros, S. (2022) · Artificial Intelligence · doi:10.1016/j.artint.2022.103840 · Cited by 63
“Moreover, since we found LORE to be computationally expensive to run (see Fig.”
MethodInfluentialStable and actionable explanations of black-box models through factual and counterfactual rules
Guidotti, R., Monreale, A., Ruggieri, S., Naretto, F., Turini, F., Pedreschi, D., & Giannotti, F. (2022) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00878-5 · Cited by 62
“We aim at advancing state-of-the-art approaches, including our previous work lore (Guidotti et al. 2019b), to a comprehensive proposal that is able to extend the coverage of comprehensibility, fidelity, minimality and generality, by also dealing with stability and actionability.”
InfluentialSelf-citationMehdiyev, N., Majlatow, M., & Fettke, P. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11399-0 · Cited by 62
Adversarial Attacks in Explainable Machine Learning: A Survey of Threats Against Models and Humans
Vadillo, J., Santana, R., & Lozano, J. A. (2024) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1567 · Cited by 59
Albahri, A. S., Hamid, R. A., Abdulnabi, A. R., Albahri, O. S., Alamoodi, A. H., Deveci, M., Pedrycz, W., Alzubaidi, L., Santamaría, J., & Gu, Y. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102208 · Cited by 55
Contestable AI Needs Computational Argumentation
Leofante, F., Ayoobi, H., Dejl, A., Freedman, G., Gorur, D., Jiang, J., Paulino-Passos, G., Rago, A., Rapberger, A., Russo, F., et al. (2024) · Principles of Knowledge Representation and Reasoning · doi:10.24963/kr.2024/83 · Cited by 56
“…that, in the case of models trained from data, redress may at times involve fine-tuning or retraining steps in which case the above methods, as well as methods for repairing AI models, e.g. (Guidotti et al. 2019a; Henriksen, Leofante, and Lomuscio 2022; Almog and Kalech 2023), could be drawn upon.”
Developing a Fidelity Evaluation Approach for Interpretable Machine LearningPreprint
Velmurugan, M., Ouyang, C., Moreira, C., & Sindhgatta, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.08492 · Cited by 53
“While methods to evaluate external fidelity are relatively common in literature (Guidotti et al, 2019; Lakkaraju et al, 2016; Ming et al, 2019; Shankaranarayana and Runje, 2019), evaluation methods to evaluate internal fidelity using black box models are generally limited to text and image data,…”
MethodAI Trust: Can Explainable AI Enhance Warranted Trust?
de Brito Duarte, R., Correia, F., Arriaga, P., & Paiva, A. (2023) · Human Behavior and Emerging Technologies · doi:10.1155/2023/4637678 · Cited by 52
Principles of Explainable Artificial Intelligence
Guidotti, R., Monreale, A., Pedreschi, D., & Giannotti, F. (2021) · Springer eBooks · doi:10.1007/978-3-030-76409-8_2 · Cited by 53
Self-citationA Roadmap of Explainable Artificial Intelligence: Explain to Whom, When, What and How?
Wang, Z., Huang, C., & Yao, X. (2024) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3702004 · Cited by 51
CASTLE: Cluster-aided space transformation for local explanations
Gatta, V. L., Moscato, V., Postiglione, M., & Sperlí, G. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115045 · Cited by 51
Model-Agnostic Counterfactual Explanations in Credit Scoring
Dastile, X., Çelik, T., & Vandierendonck, H. (2022) · IEEE Access · doi:10.1109/access.2022.3177783 · Cited by 50
“Guidotti et al. [12] proposed a method that learns a local and interpretable classifier on a synthetic neighbourhood of a record of interest (i.e. an instance that its prediction needs to be explained).”
MethodGajcin, J., & Dusparić, I. (2024) · ACM Computing Surveys · doi:10.1145/3648472 · Cited by 50
“Guidotti et al. [34] propose LORE, a local approach for generating counterfactuals.”
BackgroundMathematical optimization modelling for group counterfactual explanations
Carrizosa, E., Ramírez-Ayerbe, J., & Morales, D. R. (2024) · European Journal of Operational Research · doi:10.1016/j.ejor.2024.01.002 · Cited by 49
Feature-Weighted Counterfactual-Based Explanation for Bankruptcy Prediction
Cho, S. H., & Shin, K.-S. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.119390 · Cited by 48
Stepin, I., Alonso, J. M., Catalá, A., & Pereira-Fariña, M. (2020) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz48607.2020.9177629 · Cited by 48
“A similar approach is employed in the LOcal Rule-based Explanations (LORE) framework [21].”
MethodA Quantitative Evaluation of Global, Rule-Based Explanations of Post-Hoc, Model Agnostic Methods
Vilone, G., & Longo, L. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.717899 · Cited by 49
Carrizosa, E., Ramírez-Ayerbe, J., & Morales, D. R. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121954 · Cited by 48
“The former treat the classification model as a black-box and do not use its inner workings [Dandl et al., 2020, Guidotti et al., 2019, Poyiadzi et al., 2020], whereas the latter do.”
BackgroundAdvancing Dermatological Diagnostics: Interpretable AI for Enhanced Skin Lesion Classification
Metta, C., Beretta, A., Guidotti, R., Yin, Y., Gallinari, P., Rinzivillo, S., & Giannotti, F. (2024) · Diagnostics · doi:10.3390/diagnostics14070753 · Cited by 47
“The generation of H may follow diverse strategies, with our experiments favoring a genetic algorithm to optimize a fitness function [41].”
BackgroundMethodSelf-citationExplaining Any Time Series Classifier
Guidotti, R., Monreale, A., Spinnato, F., Pedreschi, D., & Giannotti, F. (2020) · 2020 IEEE Second International Conference on Cognitive Machine Intelligence (CogMI) · doi:10.1109/cogmi50398.2020.00029 · Cited by 46
“The proposed explanation method LASTS extends a line of research on local, model-agnostic methods originated with [20] and extended in different directions by [33] and by [25], [23].”
BackgroundMethodInfluentialSelf-citationA Critical AI View on Autonomous Vehicle Navigation: The Growing Danger
Miller, T. I., Durlik, I., Kostecka, E., Borkowski, P., & Łobodzińska, A. (2024) · Electronics · doi:10.3390/electronics13183660 · Cited by 47
Bai, S., Nasir, S., Khan, R. A., Meyer, A., & Konik, H. (2025) · IEEE Access · doi:10.1109/access.2025.3639184 · Cited by 46
“In recent times, Counterfactual explanations have been recognized as a significant post-hoc method that offers compelling insights for users to comprehend the internal workings of AI models [116, 117, 118, 119, 120].”
Computing Optimal Decision Sets with SAT
Yu, J., Ignatiev, A., Stuckey, P. J., & Bodic, P. L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58475-7_55 · Cited by 45
Counterfactual and Factual Reasoning over Hypergraphs for Interpretable Clinical Predictions on EHR.
Xu, R., Yu, Y. S., Zhang, C., Ali, M. K., Ho, J. C., & Yang, C. (2022) · PubMed · Cited by 45
Improving trust and confidence in medical skin lesion diagnosis through explainable deep learning
Metta, C., Beretta, A., Guidotti, R., Yin, Y., Gallinari, P., Rinzivillo, S., & Giannotti, F. (2023) · International Journal of Data Science and Analytics · doi:10.1007/s41060-023-00401-z · Cited by 45
Self-citationA Framework and Benchmarking Study for Counterfactual Generating Methods on Tabular Data
Oliveira, R., & Martens, D. (2021) · Applied Sciences · doi:10.3390/app11167274 · Cited by 44
“Several of these datasets are widely used in counterfactual explanation studies like Adult [53, 29, 64, 74, 21, 76, 10, 45], Statlog - German Credit [53, 21, 59, 52], Breast Cancer Wisconsin (BCW)[74, 5, 76, 4, 39] and Wine [60, 42, 4].”
MethodCapturing Users’ Reality: A Novel Approach to Generate Coherent Counterfactual Explanations
Förster, M., Hühn, P., Klier, M., & Kluge, K. (2021) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2021.155 · Cited by 43
Counterfactual explanations for misclassified images: How human and machine explanations differ
Delaney, E. D., Pakrashi, A., Greene, D., & Keane, M. T. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103995 · Cited by 43
RuleXAI—A package for rule-based explanations of machine learning model
Macha, D., Kozielski, M., Wróbel, Ł., & Sikora, M. (2022) · SoftwareX · doi:10.1016/j.softx.2022.101209 · Cited by 43
Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Box
Moreira, C., Chou, Y.-L., Hsieh, C., Ouyang, C., Pereira, J. M., & Jorge, J. A. (2025) · ACM Computing Surveys · doi:10.1145/3672553 · Cited by 43
Veldhuis, M. S., Ariëns, S., Ypma, R. J. F., Abeel, T., & Benschop, C. C. G. (2021) · Forensic Science International Genetics · doi:10.1016/j.fsigen.2021.102632 · Cited by 42
Learning Optimal Decision Sets and Lists with SAT
Yu, J., Ignatiev, A., Stuckey, P. J., & Bodic, P. L. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12719 · Cited by 41
Counterfactual Explanations for Predictive Business Process Monitoring
Huang, T.-H., Metzger, A., & Pohl, K. (2022) · Lecture notes in business information processing · doi:10.1007/978-3-030-95947-0_28 · Cited by 41
“As parameters for the genetic algorithm, we use the suggested default values from [10] with some small adjustments.”
BackgroundMethodInfluentialExplaining machine-learning models for gamma-ray detection and identification
Bandstra, M. S., Curtis, J. C., Ghawaly, J., Jones, A. C., & Joshi, T. H. Y. (2023) · PLoS ONE · doi:10.1371/journal.pone.0286829 · Cited by 39
“Instead, one may want to generate a counterfactual explanation for why one class was deemed more likely than the other, and such an explanation may reveal the features that most strongly distinguish between the two classes [41].”
BackgroundFrom Black Boxes to Conversations: Incorporating XAI in a Conversational Agent
Nguyen, V. B., Schlötterer, J., & Seifert, C. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44070-0_4 · Cited by 39
Aldeia, G. S. I., & de França, F. O. (2022) · Genetic Programming and Evolvable Machines · doi:10.1007/s10710-022-09435-x · Cited by 38
“One way to extract explanations from these kinds of models is to use an explanatory method [11–13].”
MethodUS-LIME: Increasing fidelity in LIME using uncertainty sampling on tabular data
Saadatfar, H., Kiani-Zadegan, Z., & Ghahremani-Nezhad, B. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127969 · Cited by 38
Wong, K. K. L., Han, Y., Cai, Y., Ouyang, W., Du, H., & Liu, C. (2025) · Bioengineering · doi:10.3390/bioengineering12101070 · Cited by 39
“…boundary, it yields case-level explanations; Local Rule-Based Explanations (LORE) complements this by using genetic algorithms to build a synthetic neighborhood for a locally interpretable predictor that outputs rules and counterfactuals, clarifying which factors drive a specific outcome [102,112].”
Exploring the Role of Local and Global Explanations in Recommender Systems
Radensky, M., Downey, D., Lo, K., Popović, Z., & Weld, D. S. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3519795 · Cited by 38
“Theymay be factual, explaining why a certain model outcome occurred, or contrastive, explaining why another outcome did not occur; they may also be counterfactual, explaining how another outcome could have occurred instead [18, 22, 51].”
BackgroundState-of-the-Art in Responsible, Explainable, and Fair AI for Medical Image Analysis
Amirian, S., Gao, F., Littlefield, N. G., Hill, J. H., Yates, A. J., Plate, J. F., Pantanowitz, L., Rashidi, H. H., & Tafti, A. P. (2025) · IEEE Access · doi:10.1109/access.2025.3555543 · Cited by 37
Counterfactual explanation based on gradual construction for deep networks
Jung, H.-G., Kang, S.-H., Kim, H.-D., Won, D.-O., & Lee, S. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108958 · Cited by 37
FCE: Feedback Based Counterfactual Explanations for Explainable AI
Suffian, M., Graziani, P., Alonso, J. M., & Bogliolo, A. (2022) · IEEE Access · doi:10.1109/access.2022.3189432 · Cited by 36
Achieving Diversity in Counterfactual Explanations: a Review and Discussion
Laugel, T., Jeyasothy, A., Lesot, M., Marsala, C., & Detyniecki, M. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594122 · Cited by 36
“on simultaneously exploring the data space in different directions (such as [22, 72]).”
BackgroundMethodInfluentialCounterfactual Explanation of Machine Learning Survival Models
Kovalev, M. S., Utkin, L. V., Coolen, F. P. A., & Konstantinov, A. V. (2021) · Informatica · doi:10.15388/21-infor468 · Cited by 32
“ded models is SurvLIME [38]. Counterfactual explanations. In order to get intuitive and human-friendly explanations, a lot of counterfactual explanation methods [66] were developed by several authors [9, 14, 19, 23, 24, 28, 42, 43, 49, 56, 64, 63, 71]. The counterfactual explanation tells us what to do to achieve a de…”
MethodA Combinatorial Approach to Fairness Testing of Machine Learning Models
Patel, A. R., Chandrasekaran, J., Lei, Y., Kacker, R. N., & Kuhn, D. R. (2022) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw55395.2022.00030 · Cited by 32
“A counterfactual approach identifies a minimum set of features that, if removed, shall result in a different prediction [11].”
Counterfactual explanation generation with minimal feature boundary
You, D., Niu, S., Dong, S., Yan, H., Chen, Z., Wu, D., Shen, L., & Wu, X. (2023) · Information Sciences · doi:10.1016/j.ins.2023.01.012 · Cited by 32
Understanding Any Time Series Classifier with a Subsequence-based Explainer
Spinnato, F., Guidotti, R., Monreale, A., Nanni, M., Pedreschi, D., & Giannotti, F. (2023) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3624480 · Cited by 32
“lasts adopts decision trees because factual and counterfactual decision rules can be naturally derived by following the root-leaf paths [33].”
BackgroundMethodSelf-citationMeLIME: Meaningful Local Explanation for Machine Learning ModelsPreprint
Botari, T., Hvilshøj, F., Izbicki, R., & de Carvalho, A. C. P. L. F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.05818 · Cited by 32
Alfzari, S., Al‐Shboul, M., & Alshurideh, M. T. (2025) · International Review of Management and Marketing · doi:10.32479/irmm.18594 · Cited by 31
Trustworthy AI-based Performance Diagnosis Systems for Cloud Applications: A Review
Xin, R., Wang, J., Chen, P., & Zhao, Z. (2025) · ACM Computing Surveys · doi:10.1145/3701740 · Cited by 30
“LORE [66] is a local rule-based method that generates synthetic neighborhoods around an instance, learns a decision tree to approximate their behavior, and extracts rules from the tree to explain the prediction.”
Explaining Random Forest and XGBoost with Shallow Decision Trees by Co-clustering Feature Importance
Pensa, R. G., Crombach, A., Peignier, S., & Rigotti, C. (2025) · Machine Learning · doi:10.1007/s10994-025-06932-9 · Cited by 30
“…issue has fostered a new branch of research called explainable or interpretable machine learning (Molnar, 2025), which has been the object of many surveys and review papers (Burkart & Huber, 2021; Guidotti, Monreale, Ruggieri, et al., 2019; Liang et al., 2021; Milani, Topin, Veloso, & Fang, 2024).”
InfluentialGjærum, V. B., Strümke, I., Løver, J., Miller, T., & Lekkas, A. M. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.10.014 · Cited by 29
Through the looking glass: evaluating post hoc explanations using transparent models
Velmurugan, M., Ouyang, C., Sindhgatta, R., & Moreira, C. (2023) · International Journal of Data Science and Analytics · doi:10.1007/s41060-023-00445-1 · Cited by 27
“Explainable methods that produce decision rules are also often evaluated through this approach, and the predictions of the decision rule or decision rule sets are compared against the black box model’s predictions [19, 25, 33].”
Schnake, T., Jafari, F. R., Lederer, J., Xiong, P., Nakajima, S., Gugler, S., Montavon, G., & Müller, K. (2025) · Information Fusion · doi:10.1016/j.inffus.2024.102923 · Cited by 27
Building Explanations for Fuzzy Decision Trees with the ExpliClas Software
Alonso, J. M., Ducange, P., Pecori, R., & Vilas, R. (2020) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz48607.2020.9177725 · Cited by 27
“Even though there is general purpose data mining software (e.g., Weka [11], R packages [12] or Python libraries [13]), only a few programs (e.g., LIME [14] or LORE [15]) are released as open source software for XAI.”
MethodEnhancing trust in automated 3D point cloud data interpretation through explainable counterfactuals
Holzinger, A., Lukač, N., Rozajac, D., Johnston, E., Kocic, V., Hörl, B., Gollob, C., Nothdurft, A., Stampfer, K., Schweng, S., et al. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103032 · Cited by 26
The challenges of providing explanations of AI systems when they do not behave like users expect
Riveiro, M., & Thill, S. (2022) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3503252.3531306 · Cited by 26
“Our hypothesis, given current results in the literature and examples of the use of counterfactuals [20, 42] is that hybrid explanations, i.”
ResultStepin, I., Alonso, J. M., Catalá, A., & Pereira-Fariña, M. (2022) · Information Sciences · doi:10.1016/j.ins.2022.10.098 · Cited by 26
Explaining Sentiment Classification with Synthetic Exemplars and Counter-Exemplars
Lampridis, O., Guidotti, R., & Ruggieri, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61527-7_24 · Cited by 26
“xspells, falls into the category of local, model-agnostic methods which originated with [33] and extended along diverse directions by [12] and by [14,16].”
BackgroundMethodInfluentialSelf-citationExplanation sets: A general framework for machine learning explainability
Fernández, R. R., de Diego, I. M., Moguerza, J. M., & Herrera, F. (2022) · Information Sciences · doi:10.1016/j.ins.2022.10.084 · Cited by 26
Categorical and Continuous Features in Counterfactual Explanations of AI Systems
Warren, G., Byrne, R. M. J., & Keane, M. T. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3673907 · Cited by 25
“…units”), are used as this causal connective is known to elicit causal inferences in people (e.g., [37, 67]) Many XAI researchers argue that rule-based explanations are human-interpretable as post hoc explanations for opaque models [16, 25, 51, 52, 73], although some have questioned this claim [56].”
Chowdhury, R. (2025) · doi:10.63125/x4cta041 · Cited by 25
Metadata
- Authors
- Guidotti, R., Monreale, A., Giannotti, F., Pedreschi, D., Ruggieri, S., & Turini, F.
- Affiliations
- Istituto di Scienza e Tecnologie dell'Informazione "Alessandro Faedo" (IT)
- University of Pisa (IT)
- ORCID
- Venue
- IEEE Intelligent Systems
- Published
- 2019-11-01
- Volume
- 34
- Issue
- 6
- Pages
- 14-23
- ISSN
- 1541-1672 (ISSN-L)
- 1941-1294
- OpenAlex
- W2994120362
- Cited by
- 349
- References
- 14
- Citations by year
- 2026: 38 · 2025: 67 · 2024: 62 · 2023: 60 · 2022: 47 · 2021: 48 · 2020: 26 · 2019: 1
- FWCI
- 13.92
- Type
- journal article
- Language
- en
- Open access
- green
- ieeexplore.ieee.org
- Licence
- Locations
- IEEE Intelligent Systems (publishedVersion)
- CINECA IRIS Institutial research information system (University of Pisa) (submittedVersion, other-oa)
- ISTI Open Portal (submittedVersion, other-oa)
- Scuola Normale Superiore di Pisa (submittedVersion)
- Funders
- European Commission (INFRAIA-1-2014-2015)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Data Mining Algorithms and Applications
- Keywords
- explainable AI, black-box classifiers, counterfactual explanations, genetic algorithm, interpretable classifiers, model transparency, trust in AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.