What Do You See?: Evaluation of Explainable Artificial Intelligence (XAI) Interpretability through Neural Backdoors
Open accessThe DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
EXplainable AI (XAI) methods have been proposed to interpret how a deep neural network predicts inputs through model saliency explanations that highlight the input parts deemed important to arrive at a decision for a specific target. However, it remains challenging to quantify the correctness of their interpretability as current evaluation approaches either require subjective input from humans or incur high computation cost with automated evaluation. In this paper, we propose backdoor trigger patterns--hidden malicious functionalities that cause misclassification--to automate the evaluation of saliency explanations. Our key observation is that triggers provide ground truth for inputs to evaluate whether the regions identified by an XAI method are truly relevant to its output. Since backdoor triggers are the most important features that cause deliberate misclassification, a robust XAI method should reveal their presence at inference time. We introduce three complementary metrics for the systematic evaluation of explanations that an XAI method generates. We evaluate seven state-of-the-art model-free and model-specific post-hoc methods through 36 models trojaned with specifically crafted triggers using color, shape, texture, location, and size. We found six methods that use local explanation and feature relevance fail to completely highlight trigger regions, and only a model-free approach can uncover the entire trigger region. We made our code available at https://github.com/yslin013/evalxai.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures











Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, CAM is effective in detecting the entire trigger region, but may not always provide the accurate localization of the trigger [112].”
From Survey of Explainable AI Techniques in Healthcare (2023) · cites this paper as [112]
Experimental — accuracy not yet measuredIn the citing paper
“For example, CAM is effective in detecting the entire trigger region, but may not always provide the accurate localization of the trigger [112].”
From Survey of Explainable AI Techniques in Healthcare (2023) · cites this paper as [112]
In the citing paper
“Among the popular XAI methods, backpropagation has a high rate of success in detecting and recovering from Trojan attacks, particularly for models with large trigger sizes [112].”
From Survey of Explainable AI Techniques in Healthcare (2023) · cites this paper as [112]
In the citing paper
“LIME has a high rate of success in detecting small triggers [112].”
From Survey of Explainable AI Techniques in Healthcare (2023) · cites this paper as [112]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“We use three image classification models: VGG-16 (Simonyan and Zisserman 2014), ResNet-50 (He et al. 2015) and AlexNet (Krizhevsky, Sutskever, and Hinton 2012) (Table 3 in the Appendix details models, such as their number of layers, parameters and accuracy).”
MethodVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“We use three image classification models: VGG-16 (Simonyan and Zisserman 2014), ResNet-50 (He et al.”
MethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 7×
“We trojan three image classification models, VGG-16 [36], ResNet50 [13] and AlexNet [17], through poisoning attack (Algorithm 1).”
InfluentialYou Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · cited 1×
“To handle multiple triggers, we plan to use object detection algorithms such as YOLO (Redmon et al. 2016) to capture multiple objects highlighted by the XAI methods.”
MethodImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 1×
“We evaluate seven XAI methods on 18 single target and 18multiple target trojaned models with the ImageNet dataset [31], which consists of one million images with 1,000 classes.”
MethodLearning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · cited 1×
“, stamped with trigger) input x′, a model F′ is trained by solving a supervised learning problem through backpropagation [30], where x and x′ are classified to y (true label) and yt (target label) respectively.”
MethodPyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · cited 1×
“sion on the performance of each XAI method on trojaned models through introduced evaluation metrics. Lastly, we compare the computation cost of XAI methods. We conducted our experiments with PyTorch (Paszke et al. 2019) using NVIDIA Tesla T4 GPU and four vCPU with 26 GB of memory provided by the Google Cloud platform.…”
MethodInfluentialA Computational Approach to Edge Detection
Canny, J. (1986) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1986.4767851 · Cited by 30,004 · cited 2×
“2), we leverage the popular Canny (Canny 1986) edge detection algorithm to identify the most salient region and draw a bounding box to cover the detected pixels.”
MethodGrad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 13×
“We found that forwardbased XAI methods (OCC, FA and LIME) gives better metrics for small triggers, and LIME outperforms the other XAI methods for eight out of 18 trojaned models.”
MethodInfluentialLearning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428 · cited 1×
“ative gradients are set to zero–during the backpropagation. (3) Gradient-weighted Class Activation Mapping(GCAM) (Selvaraju et al. 2017) is a relaxed generalization of Class Activation Mapping (CAM) (Zhou et al. 2016), which produces a coarse localization map by upsampling a linear combination of features in the last…”
BackgroundArrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321 · cited 1×
“With the popularity of DNN applications, numerous XAI methods have been proposed to reason about the decision of a model for a given input (Arrieta et al. 2020; Adadi and Berrada 2018).”
MethodBarredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 1×
“With the popularity of DNN applications, numerous XAI methods have been proposed to reason about the decision of a model for a given input [1, 3].”
MethodDeep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 2×
“(1) Backpropagation (BP) [35] uses the gradients of the input layer with respect to the prediction result to render a normalized heatmap for deriving important features as interpretation.”
MethodResultGrad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks
Chattopadhay, A., Sarkar, A., Howlader, P., & Balasubramanian, V. N. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00097 · Cited by 3,639 · cited 1×
“Other works [6, 34] leverage dataset with manually-marked bounding boxes to evaluate their interpretability results.”
BackgroundStriving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447 · cited 2×
“g XAI frameworks, the assessment of model interpretability can be done with human interventions. For example, previous works (Ribeiro, Singh, and Guestrin 2016; Simonyan, Vedaldi, and Zisserman 2013; Springenberg et al. 2014; Kim et al. 2018) require human assistance for judgment of XAI method results. Other works (Ch…”
BackgroundMethodSanity Checks for Saliency Maps
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294 · cited 1×
“that XAI methods may produce similar results between normally trained models and models trained using randomized inputs (i.e., when the relationships between inputs and output labels are randomized) (Adebayo et al. 2018). They also yield largely different results with input perturbations (Ghorbani, Abid, and Zou 2017;…”
ResultTrojaning Attack on Neural Networks
Liu, Y., Ma, S., Aafer, Y., Lee, W.-C., Zhai, J., Wang, W., & Zhang, X. (2018) · Proceedings 2018 Network and Distributed System Security Symposium · doi:10.14722/ndss.2018.23291 · Cited by 1,270 · cited 2×
“ined models (See Table 3 in the Appendix). Additionally, models with triggers of larger sizes located at fixed positions yield higher MR, which is consistent with the observation of the previous work (Liu et al. 2018). Multiple Target Trojaned models. We additionally trojan each model with eight target labels using tri…”
BackgroundResultSelf-citationKim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., & Sayres, R. (2017) · arXiv · doi:10.48550/arxiv.1711.11279 · Cited by 1,240 · cited 1×
“ssment of model interpretability can be done with human interventions. For example, previous works (Ribeiro, Singh, and Guestrin 2016; Simonyan, Vedaldi, and Zisserman 2013; Springenberg et al. 2014; Kim et al. 2018) require human assistance for judgment of XAI method results. Other works (Chattopadhay et al. 2018; Se…”
BackgroundInterpretable Explanations of Black Boxes by Meaningful Perturbation
Fong, R. C., & Vedaldi, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.371 · Cited by 1,097 · cited 1×
“There exist automatic XAI method evaluation methods through inspecting accuracy degradation by masking or perturbing the most relevant region (Samek et al. 2016; Fong and Vedaldi 2017; Ancona et al. 2017; Melis and Jaakkola 2018; Yeh et al. 2019).”
MethodBadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
Gu, T., Dolan-Gavitt, B., & Garg, S. (2017) · arXiv · doi:10.48550/arxiv.1708.06733 · Cited by 1,061 · cited 4×
“The first trojan attack trains a backdoored DNN model with data poisoning using images with a trigger attached and labeled as the specified target label [7, 12].”
BackgroundMethodInfluentialTargeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
Chen, X., Liu, C., Li, B., Lu, K., & Song, D. (2017) · arXiv · doi:10.48550/arxiv.1712.05526 · Cited by 1,039 · cited 3×
“Poisoning attacks (Chen et al. 2017; Gu, Dolan-Gavitt, and Garg 2017) involves adversaries that train a target neural network with a dataset consisting of normal and poisoned (i.”
BackgroundMethodInfluentialA Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785 · cited 2×
“The research community recently proposed different evaluation metrics other than evaluating the correctness of XAI methods [25], such as generalizability, persuasibility, and fidelity, to evaluate XAI properties.”
MethodA Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2018) · arXiv · doi:10.48550/arxiv.1811.11839 · Cited by 783
Visualizing and Understanding Convolutional Networks
Zeiler, M. D., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1311.2901 · Cited by 522 · cited 13×
“We found that forwardbased XAI methods (OCC, FA and LIME) gives better metrics for small triggers, and LIME outperforms the other XAI methods for eight out of 18 trojaned models.”
MethodInfluentialEvaluating XAI: A comparison of rule-based and example-based explanations
van der Waa, J., Nieuwburg, E., Cremers, A., & Neerincx, M. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2020.103404 · Cited by 393 · cited 1×
“For example, previous works [15, 16, 24, 26, 29, 35, 37] require human assistance for judgment of XAI method results.”
ResultTowards better understanding of gradient-based attribution methods for Deep Neural Networks
Ancona, M., Ceolini, E., Öztireli, C., & Groß, M. H. (2017) · Zurich Open Repository and Archive (University of Zurich) · doi:10.3929/ethz-b-000249929 · Cited by 368 · cited 1×
“There exist automated XAI method evaluation methods through inspecting accuracy degradation by masking or perturbing the most relevant region [2, 11, 22, 33, 42].”
MethodA Benchmark for Interpretability Methods in Deep Neural Networks
Hooker, S., Erhan, D., Kindermans, P.-J., & Kim, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.10758 · Cited by 365 · cited 2×
“iffer in size, location, color, and texture in order to check that the identified regions by XAI methods are truly relevant to the output label. Our approach eliminates the distribution shift problem (Hooker et al. 2019), requires no model retraining processes, and applicable to evaluate any type of XAI methods. We stu…”
MethodLatent Backdoor Attacks on Deep Neural Networks
Yao, Y., Li, H., Zheng, H., & Zhao, B. Y. (2019) · Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3319535.3354209 · Cited by 343 · cited 1×
“Various triggers are developed by leveraging these approaches, such as transferred [12, 41], perturbation [19], and invisible triggers [18, 32].”
BackgroundProxy tasks and subjective measures can be misleading in evaluating explainable AI systems
Buçinca, Z., Lin, P., Gajos, K. Z., & Glassman, E. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377498 · Cited by 307 · cited 1×
“However, human subjective measurements are tedious and time-consuming and may introduce bias and produce inaccurate evaluations [4].”
BackgroundSentiNet: Detecting Localized Universal Attacks Against Deep Learning Systems
Chou, E., Tramer, F., & Pellegrino, G. (2020) · 2020 IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw50608.2020.00025 · Cited by 281 · cited 1×
“This means using XAI methods for input purification against trojan attack [8, 9] is Figure 8: The forward-based methods (OCC, FA and LIME) in-”
MethodFebruus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems
Doan, B. G., Abbasnejad, E., & Ranasinghe, D. C. (2020) · Annual Computer Security Applications Conference · doi:10.1145/3427228.3427264 · Cited by 263
Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
Zhong, H., Liao, C., Squicciarini, A. C., Zhu, S., & Miller, D. (2020) · Proceedings of the Tenth ACM Conference on Data and Application Security and Privacy · doi:10.1145/3374664.3375751 · Cited by 180 · cited 1×
“ion to different target labels (Tang et al. 2020). Various triggers are developed by leveraging these approaches, such as transferred (Gu, Dolan-Gavitt, and Garg 2017; Yao et al. 2019), perturbation (Liao et al. 2018), and invisible triggers (Li et al. 2019; Saha, Subramanya, and Pirsiavash 2019). Interpretability of…”
BackgroundAblation Studies in Artificial Neural Networks
Meyes, R., Lu, M., de Puiseau, C. W., & Meisen, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08644 · Cited by 168 · cited 1×
“(6) Feature Ablation(FA) (Meyes et al. 2019) splits input features into several groups, where each group is perturbed together to determine the importance of each group by ob-”
BackgroundAn Embarrassingly Simple Approach for Trojan Attack in Deep Neural Networks
Tang, R., Du, M., Liu, N., Yang, F., & Hu, X. (2020) · Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3394486.3403064 · Cited by 165 · cited 1×
“The last approach integrates a trojan module into the target model, which combines the output of two networks for triggers that causes misclassification to different target labels [38].”
BackgroundEvaluating Explanation Methods for Deep Learning in Security
Warnecke, A., Arp, D., Wressnegger, C., & Rieck, K. (2020) · 2020 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp48549.2020.00018 · Cited by 127 · cited 1×
“As [3] stated, these metrics are often either conceptual without quantitative measures such as the metrics proposed in [40] or are domain-specific focusing on malware detection and vulnerability discovery [39].”
BackgroundWhen Explainability Meets Adversarial Learning: Detecting Adversarial Examples using SHAP Signatures
Fidel, G., Bitton, R., & Shabtai, A. (2020) · 2020 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207637 · Cited by 121 · cited 1×
“For example, one family of XAI methods target on interpretability aims to describe the internal workings of a neural network in a way that is understandable to humans, which inspired works such as model debugging and adversarial input detection [10, 43] that leverage saliency explanations provided by these methods.”
Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
Liao, C., Zhong, H., Squicciarini, A., Zhu, S., & Miller, D. (2018) · arXiv · doi:10.48550/arxiv.1808.10307 · Cited by 110 · cited 1×
“ion to different target labels (Tang et al. 2020). Various triggers are developed by leveraging these approaches, such as transferred (Gu, Dolan-Gavitt, and Garg 2017; Yao et al. 2019), perturbation (Liao et al. 2018), and invisible triggers (Li et al. 2019; Saha, Subramanya, and Pirsiavash 2019). Interpretability of…”
BackgroundLiu, Y., Mondal, A., Chakraborty, A., Zuzak, M., Jacobsen, N., Xing, D., & Srivastava, A. (2020) · 2020 21st International Symposium on Quality Electronic Design (ISQED) · doi:10.1109/isqed48828.2020.9137011 · Cited by 68 · cited 1×
“We evaluate the interpretability of XAI methods by checking whether they can detect backdoor triggers (Liu et al. 2020) present in the input, which cause a trojaned model to output a spear X iv :2 00 9.”
MethodExpert Level Evaluations for Explainable AI (XAI) Methods in the Medical Domain
Muddamsetty, S. M., Jahromi, M. N. S., & Moeslund, T. B. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68796-0_3 · Cited by 62 · cited 1×
“For example, previous works [15, 16, 24, 26, 29, 35, 37] require human assistance for judgment of XAI method results.”
ResultOn the (In)fidelity and Sensitivity for Explanations
Yeh, C., Hsieh, C.-Y., Suggala, A. S., Inouye, D. I., & Ravikumar, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09392 · Cited by 60 · cited 1×
“There exist automatic XAI method evaluation methods through inspecting accuracy degradation by masking or perturbing the most relevant region (Samek et al. 2016; Fong and Vedaldi 2017; Ancona et al. 2017; Melis and Jaakkola 2018; Yeh et al. 2019).”
MethodA Human-Grounded Evaluation Benchmark for Local Explanations of Machine Learning
Mohseni, S., Block, J. E., & Ragan, E. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.05075 · Cited by 45 · cited 1×
“For example, previous works [15, 16, 24, 26, 29, 35, 37] require human assistance for judgment of XAI method results.”
ResultInvisible Backdoor Attacks Against Deep Neural Networks
Li, S., Zhao, B. Z. H., Yu, J., Xue, M., Kaafar, D., & Zhu, H. (2019) · Cited by 37
Evaluating the visualization of what a Deep Neural Network has learnedPreprint
Samek, W., Binder, A., Montavon, G., Bach, S., & Müller, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.06321 · Cited by 35 · cited 1×
“There exist automatic XAI method evaluation methods through inspecting accuracy degradation by masking or perturbing the most relevant region (Samek et al. 2016; Fong and Vedaldi 2017; Ancona et al. 2017; Melis and Jaakkola 2018; Yeh et al. 2019).”
MethodEvaluating Explanation Without Ground Truth in Interpretable Machine Learning
Yang, F., Du, M., & Hu, X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.06831 · Cited by 34 · cited 1×
“As [3] stated, these metrics are often either conceptual without quantitative measures such as the metrics proposed in [40] or are domain-specific focusing on malware detection and vulnerability discovery [39].”
BackgroundTowards better understanding of gradient-based attribution methods for Deep Neural NetworksPreprint
Ancona, M., Ceolini, E., Öztireli, C., et al. (2017) · Zurich Open Repository and Archive (University of Zurich) · doi:10.48550/arxiv.1711.06104 · Cited by 26
Hidden Trigger Backdoor Attacks
Saha, A., Subramanya, A., & Pirsiavash, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6871 · Cited by 13
Research works printed but not linked: 1
- David Alvarez Melis and Tommi Jaakkola. 2018. Towards robust interpretabil- ity with self-explaining neural networks. In Advances in Neural Information Processing Systems. 7775-7784.
Cited by
Li, Y., Jiang, Y., Li, Z., & Xia, S.-T. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3182979 · Cited by 682
“Most recently, backdoor attacks were also adopted for interpreting DNNs [111] and the evaluation of explainable AI methods [112].”
BackgroundSurvey of Explainable AI Techniques in Healthcare
Chaddad, A., Peng, J., Xu, J., & Bouridane, A. (2023) · Sensors · doi:10.3390/s23020634 · Cited by 591
Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond
Li, X., Xiong, H., Li, X., Wu, X., Zhang, X., Liu, J., Bian, J., & Dou, D. (2022) · Knowledge and Information Systems · doi:10.1007/s10115-022-01756-8 · Cited by 497
“[106] evaluated the trustworthiness of interpretation algorithms.”
BackgroundSecurity and Privacy Challenges of Large Language Models: A Survey
Das, B. C., Amini, M. H., & Wu, Y. (2025) · ACM Computing Surveys · doi:10.1145/3712001 · Cited by 445
Nazar, M., Alam, M. M., Yafi, E., & Su'ud, M. M. (2021) · IEEE Access · doi:10.1109/access.2021.3127881 · Cited by 312
Metadata
- Authors
- Lin, Y.-S., Lee, W.-C., & Celik, Z. B.
- Affiliations
- Purdue University West Lafayette (US)
- Apple (United States) (US)
- ORCID
- Lee, Wen-Chuan
- Venue
- Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining
- Publisher
- ACM
- Published
- 2021-08-14
- Pages
- 1027-1035
- OpenAlex
- W3170542759
- Cited by
- 64
- References
- 40
- Citations by year
- 2026: 4 · 2025: 22 · 2024: 16 · 2023: 15 · 2022: 6 · 2021: 1
- FWCI
- 4.39
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- explainable AI, saliency maps, backdoor triggers, backdoor attacks, explanation evaluation, post-hoc explanation methods, adversarial robustness, misclassification, trigger detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.