Cited by 515Explainable Artificial Intelligence (XAI)Adversarial Robustness in Machine LearningSoftware Testing and Debugging Techniques

The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.

Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations

Open access

Ross, Andrew Slavin, Hughes, Michael C., Doshi-Velez, Finale·2017·Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence

DOI 10.24963/ijcai.2017/371

Open this paper in CoChat — free account Open PDF

Abstract

Expressive classifiers such as neural networks are among the most accurate supervised learning methods in use today, but their opaque decision boundaries make them difficult to trust in critical applications. We propose a method to explain the predictions of any differentiable model via the gradient of the class label with respect to the input (which provides a normal to the decision boundary). Not only is this approach orders of magnitude faster at identifying input dimensions of high sensitivity than sample-based perturbation methods (e.g. LIME), but it also lends itself to efficiently discovering multiple qualitatively different decision boundaries as well as decision boundaries that are consistent with expert annotation. On multiple datasets, we show our approach generalizes much better when test conditions differ from those in training.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.24963/ijcai.2017/371

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 24,190
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery.
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524
Fairness through awareness
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks
Papernot, N., McDaniel, P. D., Wu, X., Jha, S., & Swami, A. (2016) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2016.41 · Cited by 3,251
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
Self-citation

Show all 31 references

Research works printed but not linked: 4

David Baehrens, Timon Schroeter, Stefan Harmeling, Motoaki Kawanabe, Katja Hansen, and Klaus-Robert Müller. How to explain individual classification decisions. Journal of Machine Learning Research, 11(Jun):1803-1831, 2010.
Fong and Vedaldi, 2017] Ruth Fong and Andrea Vedaldi. Inter- pretable explanations of black boxes by meaningful perturbation. arXiv preprint arXiv:1704.03296, 2017. · arXiv
Tao Lei, Regina Barzilay, and Tommi Jaakkola. Ra- tionalizing neural predictions. arXiv preprint arXiv:1606.04155, 2016. · arXiv
Zachary C Lipton. The mythos of model inter- pretability. arXiv preprint arXiv:1606.03490, 2016. · arXiv

Sorted by citations

Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
“While there are several different techniques for obtaining local-explanations [96]–[100], recent work by Lundberg and Lee [101] have shown that there are equivalences among these techniques.”
Method
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507
Definitions, methods, and applications in interpretable machine learning
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900654116 · Cited by 2,167
“Given the challenges surrounding simply generating post hoc interpretations, research on their uses has been limited (97, 98).”
Background
Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3060483 · Cited by 1,477
COVID-19 Image Data Collection: Prospective Predictions are the Future
Cohen, J., Morrison, P., Dao, L., Roth, K., Duong, T. Q., & Ghassemi, M. (2020) · The Journal of Machine Learning for Biomedical Imaging · doi:10.59275/j.melba.2020-48g7 · Cited by 1,110
Attention is not not Explanation
Wiegreffe, S., & Pinter, Y. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1002 · Cited by 966
Attention is not Explanation
Jain, S., & Wallace, B. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1357 · Cited by 1,133
Interpretable Convolutional Neural Networks
Zhang, Q., Wu, Y. N., & Zhu, S.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00920 · Cited by 833
Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems
Yu, J., Lu, L., Meng, X., & Karniadakis, G. E. (2022) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2022.114823 · Cited by 737
Is Attention Interpretable?
Serrano, S., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1282 · Cited by 690
Towards Explainable Artificial Intelligence
Samek, W., & Müller, K. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-28954-6_1 · Cited by 704
Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing Their Input Gradients
Ross, A. S., & Doshi‐Velez, F. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11504 · Cited by 647
Self-citation
Explainable machine learning in deployment
Bhatt, U., Xiang, A., Sharma, S., Weller, A., Taly, A., Jia, Y., Ghosh, J., Puri, R., Moura, J. M. F., & Eckersley, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375624 · Cited by 633
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785
On Interpretability of Artificial Neural Networks: A Survey
Fan, F., Xiong, J., Li, M., & Wang, G. (2021) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2021.3066428 · Cited by 607
From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI
Nauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schlötterer, J., van Keulen, M., & Seifert, C. (2023) · ACM Computing Surveys · doi:10.1145/3583558 · Cited by 590
Attention is not ExplanationPreprint
Jain, S., & Wallace, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.10186 · Cited by 897
Interpretability of deep learning models: A survey of results
Chakraborty, S., Tomsett, R., Raghavendra, R., Harborne, D., Alzantot, M., Cerutti, F., Srivastava, M. B., Preece, A. D., Julier, S., Rao, R., et al. (2017) · IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) · doi:10.1109/uic-atc.2017.8397411 · Cited by 502
Accountability of AI Under the Law: The Role of Explanation
Doshi-Velez, F., Kortz, M., Budish, R., Bavitz, C., Gershman, S. J., O'Brien, D., Scott, K., Shieber, S., Waldo, J., Weinberger, D., et al. (2017) · doi:10.2139/ssrn.3064761 · Cited by 457
Self-citation
A Research Agenda for Hybrid Intelligence: Augmenting Human Intellect With Collaborative, Adaptive, Responsible, and Explainable Artificial Intelligence
Akata, Z., Balliet, D., de Rijke, M., Dignum, F., Dignum, V., Eiben, G., Fokkens, A., Grossi, D., Hindriks, K., Hoos, H., et al. (2020) · Computer · doi:10.1109/mc.2020.2996587 · Cited by 415
Women Also Snowboard: Overcoming Bias in Captioning Models
Hendricks, L. A., Burns, K., Saenko, K., Darrell, T., & Rohrbach, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_47 · Cited by 404
Deterministic Local Interpretable Model-Agnostic Explanations for Stable Explainability
Zafar, M. R., & Khan, N. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3030027 · Cited by 432
Counterfactual Samples Synthesizing for Robust Visual Question Answering
Chen, L., Yan, X., Xiao, J., Zhang, H., Pu, S., & Zhuang, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01081 · Cited by 361
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes
Hsieh, C.-Y., Li, C.-L., Yeh, C.-K., Nakhost, H., Fujii, Y., Ratner, A., Krishna, R., Lee, C.-Y., & Pfister, T. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.507 · Cited by 316
Binder 2.0 - Reproducible, interactive, sharable environments for science at scale
Project Jupyter, Bussonnier, M., Forde, J. Z., Freeman, J., Granger, B., Head, T., Holdgraf, C., Kelley, K., Nalvarte, G., Osheroff, A., et al. (2018) · Proceedings of the Python in Science Conferences · doi:10.25080/majora-4af1f417-011 · Cited by 303
Improving performance of deep learning models with axiomatic attribution priors and expected gradients
Erion, G., Janizek, J. D., Sturmfels, P., Lundberg, S. M., & Lee, S.-I. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00343-w · Cited by 300
Large pre-trained language models contain human-like biases of what is right and wrong to do
Schramowski, P., Turan, C., Andersen, N., Rothkopf, C. A., & Kersting, K. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00458-8 · Cited by 291
Neural Prototype Trees for Interpretable Fine-grained Image Recognition
Nauta, M., van Bree, R., & Seifert, C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01469 · Cited by 269
Predicting COVID-19 Pneumonia Severity on Chest X-ray With Deep Learning
Cohen, J., Dao, L., Roth, K., Morrison, P., Bengio, Y., Abbasi, A., Shen, B., Mahsa, H. K., Ghassemi, M., Li, H., et al. (2020) · Cureus · doi:10.7759/cureus.9448 · Cited by 261
A Survey of the State of Explainable AI for Natural Language Processing
Danilevsky, M., Qian, K., Aharonov, R., Katsis, Y., Kawas, B., & Sen, P. (2020) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (AACL-IJCNLP) · doi:10.18653/v1/2020.aacl-main.46 · Cited by 468
Seq2seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models
Strobelt, H., Gehrmann, S., Behrisch, M., Perer, A., Pfister, H., & Rush, A. M. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2865044 · Cited by 265
Fairness in Deep Learning: A Computational Perspective
Du, M., Yang, F., Zou, N., & Hu, X. (2020) · IEEE Intelligent Systems · doi:10.1109/mis.2020.3000681 · Cited by 218
Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI
Mueller, S. T., Hoffman, R. R., Clancey, W., Emrey, A., & Klein, G. (2019) · arXiv · doi:10.48550/arxiv.1902.01876 · Cited by 215
Beyond Sparsity: Tree Regularization of Deep Models for Interpretability
Wu, M., Hughes, M., Parbhoo, S., Zazzi, M., Roth, V., & Doshi-Velez, F. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11501 · Cited by 213
Self-citation
Explanatory Interactive Machine Learning
Teso, S., & Kersting, K. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314293 · Cited by 226
Making deep neural networks right for the right scientific reasons by interacting with their explanations
Schramowski, P., Stammer, W., Teso, S., Brugger, A., Herbert, F., Shao, X., Luigs, H.-G., Mahlein, A.-K., & Kersting, K. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0212-3 · Cited by 207
A Review on Explainability in Multimodal Deep Neural Nets
Joshi, G., Walambe, R., & Kotecha, K. (2021) · IEEE Access · doi:10.1109/access.2021.3070212 · Cited by 204
Visualizing the Impact of Feature Attribution Baselines
Sturmfels, P., Lundberg, S., & Lee, S. (2020) · Distill · doi:10.23915/distill.00022 · Cited by 249
Detecting and interpreting myocardial infarction using fully convolutional neural networks
Strodthoff, N., & Strodthoff, C. (2018) · Physiological Measurement · doi:10.1088/1361-6579/aaf34d · Cited by 219
CLEVR-XAI: A benchmark dataset for the ground truth evaluation of neural network explanations
Arras, L., Osman, A., & Samek, W. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.11.008 · Cited by 226
Human-Centered Artificial Intelligence for Designing Accessible Cultural Heritage
Pisoni, G., Díaz-Rodríguez, N., Gijlers, H., & Tonolli, L. (2021) · Applied Sciences · doi:10.3390/app11020870 · Cited by 183
Considerations for Evaluation and Generalization in Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2018) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-319-98131-4_1 · Cited by 176
Self-citation
Interpretable and explainable machine learning: A methods‐centric overview with concrete examples
Marcinkevičs, R., & Vogt, J. E. (2023) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1493 · Cited by 210
Explainable Generative AI (GenXAI): a survey, conceptualization, and research agenda
Schneider, J. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10916-x · Cited by 214
Asking ‘Why’ in AI: Explainability of intelligent systems – perspectives and challenges
Preece, A. D. (2018) · Intelligent systems in accounting, finance and management/Intelligent systems in accounting, finance & management · doi:10.1002/isaf.1422 · Cited by 169
Beyond explaining: Opportunities and challenges of XAI-based model improvement
Weber, L., Lapuschkin, S., Binder, A., & Samek, W. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.11.013 · Cited by 168
Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems
Tomsett, R., Braines, D., Harborne, D., Preece, A. D., & Chakraborty, S. (2018) · ORCA Online Research @Cardiff (Cardiff University) · doi:10.48550/arxiv.1806.07552 · Cited by 127
Differential effects of cognitive training modules in healthy aging and mild cognitive impairment: A comprehensive meta-analysis of randomized controlled trials.
Basak, C., Qin, S., & O'Connell, M. A. (2020) · Psychology and Aging · doi:10.1037/pag0000442 · Cited by 138
EDITS: Modeling and Mitigating Data Bias for Graph Neural Networks
Dong, Y., Liu, N., Jalaian, B. A., & Li, J. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512173 · Cited by 125
Invariant Grounding for Video Question Answering
Li, Y., Wang, X., Xiao, J., Ji, W., & Chua, T. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00294 · Cited by 132
Finding and removing Clever Hans: Using explanation methods to debug and improve deep models
Anders, C. J., Weber, L., Neumann, D., Samek, W., Müller, K., & Lapuschkin, S. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.07.015 · Cited by 145
An Evaluation of the Human-Interpretability of Explanation
Lage, I., Chen, E., He, J., Narayanan, M., Kim, B., Gershman, S., & Doshi-Velez, F. (2019) · arXiv · doi:10.48550/arxiv.1902.00006 · Cited by 125
Self-citation
Deep neural networks for choice analysis: Extracting complete economic information for interpretation
Wang, S., Wang, Q., & Zhao, J. (2020) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2020.102701 · Cited by 127
Who needs to know what, when?: Broadening the Explainable AI (XAI) Design Space by Looking at Explanations Across the AI Lifecycle
Dhanorkar, S., Wolf, C. T., Qian, K., Xu, A., Popa, L., & Li, Y. (2021) · Designing Interactive Systems Conference 2021 · doi:10.1145/3461778.3462131 · Cited by 119
Women also Snowboard: Overcoming Bias in Captioning ModelsPreprint
Hendricks, L. A., Burns, K., Saenko, K., Darrell, T., & Rohrbach, A. (2018) · European Conference on Computer Vision · doi:10.48550/arxiv.1803.09797 · Cited by 536
A Perspective on Explanations of Molecular Prediction Models
Wellawatte, G. P., Gandhi, H. A., Seshadri, A., & White, A. D. (2023) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.2c01235 · Cited by 119
A Survey of the State of Explainable AI for Natural Language ProcessingPreprint
Danilevsky, M., Qian, K., Aharonov, R., Katsis, Y., Kawas, B., & Sen, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.00711 · Cited by 108
RelEx: A Model-Agnostic Relational Model Explainer
Zhang, Y., DeFazio, D., & Ramesh, A. (2020) · AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462562 · Cited by 141
Detecting Statistical Interactions from Neural Network WeightsPreprint
Tsang, M., Cheng, D., & Liu, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.04977 · Cited by 161
Clinical Predictive Models for COVID-19: Systematic Study
Schwab, P., Schütte, A. D., Dietz, B., & Bauer, S. (2020) · Journal of Medical Internet Research · doi:10.2196/21439 · Cited by 112
Data-Centric Explanations: Explaining Training Data of Machine Learning Systems to Promote Transparency
Anik, A. I., & Bunt, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445736 · Cited by 103
Compositional Convolutional Neural Networks: A Robust and Interpretable Model for Object Recognition Under Occlusion
Kortylewski, A., Liu, Q., Wang, A., Sun, Y., & Yuille, A. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-020-01401-3 · Cited by 98
Incorporating Priors with Feature Attribution on Text Classification
Liu, F., & Avci, B. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1631 · Cited by 120
Examining CNN Representations With Respect to Dataset Bias
Zhang, Q., Wang, W., & Zhu, S. C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11833 · Cited by 95
EnD: Entangling and Disentangling deep representations for bias correction
Tartaglione, E., Barbano, C. A., & Grangetto, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01330 · Cited by 92
Explainable AI: A review of applications to neuroimaging data
Farahani, F. V., Fiok, K., Lahijanian, B., Karwowski, W., & Douglas, P. K. (2022) · Frontiers in Neuroscience · doi:10.3389/fnins.2022.906290 · Cited by 87
A survey on artificial intelligence in histopathology image analysis
Abdelsamea, M. M., Zidan, U., Senousy, Z., Gaber, M. M., Rakha, E. A., & Ilyas, M. (2022) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1474 · Cited by 84
Modeling and interpreting hydrological responses of sustainable urban drainage systems with explainable machine learning methods
Yang, Y., & Chui, T. F. M. (2021) · Hydrology and earth system sciences · doi:10.5194/hess-25-5839-2021 · Cited by 81
The Role of Explainability in Assuring Safety of Machine Learning in Healthcare
Jia, Y., McDermid, J. A., Lawton, T., & Habli, I. (2022) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2022.3171314 · Cited by 93
Evaluating Attribution for Graph Neural Networks
Sánchez-Lengeling, B., Wei, J. N., Lee, B. K., Reif, E., Wang, P. Y., Qian, W. W., McCloskey, K., Colwell, L. J., & Wiltschko, A. B. (2020) · neural information processing systems · Cited by 75
Debugging Tests for Model Explanations
Adebayo, J., Muelly, M., Liccardi, I., & Kim, B. (2020) · arXiv · doi:10.48550/arxiv.2011.05429 · Cited by 88
Interpretable CNNs for Object Classification
Zhang, Q., Wang, X., Wu, Y. N., Zhou, H., & Zhu, S. C. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2982882 · Cited by 78
Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations
Stammer, W., Schramowski, P., & Kersting, K. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00362 · Cited by 73
The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanations
Balagopalan, A., Zhang, H., Hamidieh, K., Hartvigsen, T., Rudzicz, F., & Ghassemi, M. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533179 · Cited by 98
Going Beyond XAI: A Systematic Survey for Explanation-Guided Learning
Gao, Y., Gu, S., Jiang, J., Hong, S.-S., Yu, D., & Zhao, L. (2024) · ACM Computing Surveys · doi:10.1145/3644073 · Cited by 85
A multi-institutional study using artificial intelligence to provide reliable and fair feedback to surgeons
Kiyasseh, D., Laca, J., Haque, T. F., Miles, B. J., Wagner, C., Donoho, D. A., Anandkumar, A., & Hung, A. J. (2023) · Communications Medicine · doi:10.1038/s43856-023-00263-3 · Cited by 69
Explaining A Black-box By Using A Deep Variational Information Bottleneck Approach
Bang, S., Xie, P., Lee, H., Wu, W., & Xing, E. P. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17358 · Cited by 73
Problems in the deployment of machine-learned models in health care
Cohen, J., Cao, T., Viviano, J. D., Huang, C.-W., Fralick, M., Ghassemi, M., Mamdani, M., Greiner, R., & Bengio, Y. (2021) · Canadian Medical Association Journal · doi:10.1503/cmaj.202066 · Cited by 70
A systematic review of generalization research in medical image classification
Matta, S. C., Lamard, M., Zhang, P., Guilcher, A. L., Borderie, L., Cochener, B., & Quellec, G. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109256 · Cited by 83
Toward Interpretable Machine Learning: Transparent Deep Neural Networks and BeyondPreprint
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Müller, K. (2020) · arXiv (Cornell University) · Cited by 70
A Survey of Evaluation Methods and Measures for Interpretable Machine LearningPreprint
Mohseni, S., Zarei, N., & Ragan, E. D. (2018) · arXiv (Cornell University) · Cited by 69
Improving deep neural network generalization and robustness to background bias via layer-wise relevance propagation optimization
Bassi, P. R. A. S., Dertkigil, S. S. J., & Cavalli, A. (2024) · Nature Communications · doi:10.1038/s41467-023-44371-z · Cited by 80
Uncovering and Correcting Shortcut Learning in Machine Learning Models for Skin Cancer Diagnosis
Nauta, M., Walsh, R., Dubowski, A., & Seifert, C. (2021) · Diagnostics · doi:10.3390/diagnostics12010040 · Cited by 67
Towards relatable explainable AI with the perceptual process
Zhang, W., & Lim, B. Y. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501826 · Cited by 67
Transformer based contextualization of pre-trained word embeddings for irony detection in Twitter
González, J. Á., Hurtado, L.-F., & Plà, F. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2020.102262 · Cited by 69
Dissonance Between Human and Machine Understanding
Zhang, Z., Singh, J., Gadiraju, U., & Anand, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359158 · Cited by 62
A review of data-driven modelling in drinking water treatment
Aliashrafi, A., Zhang, Y., Groenewegen, H., & Peleato, N. M. (2021) · Reviews in Environmental Science and Bio/Technology · doi:10.1007/s11157-021-09592-y · Cited by 59
Testing and verification of neural-network-based safety-critical control software: A systematic literature review
Zhang, J., & Li, J. (2020) · Information and Software Technology · doi:10.1016/j.infsof.2020.106296 · Cited by 66
Assessing Heuristic Machine Learning Explanations with Model Counting
Narodytska, N., Shrotri, A. A., Meel, K. S., Ignatiev, A., & Marques-Silva, J. (2019) · International Conference on Theory and Applications of Satisfiability Testing · doi:10.1007/978-3-030-24258-9_19 · Cited by 68
Evaluating Feature Importance EstimatesPreprint
Hooker, S., Erhan, D., Kindermans, P.-J., & Kim, B. (2018) · arXiv (Cornell University) · Cited by 60
Learning Explainable Models Using Attribution Priors
Erion, G., Janizek, J. D., Sturmfels, P., Lundberg, S., & Lee, S. (2019) · Cited by 53
Counterfactual Samples Synthesizing and Training for Robust Visual Question Answering
Chen, L., Zheng, Y., Niu, Y., Zhang, H., & Xiao, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3290012 · Cited by 57
Leveraging explanations in interactive machine learning: An overview
Teso, S., Alkan, Ö., Stammer, W., & Daly, E. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1066049 · Cited by 76
Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers
Chen, H., & Ji, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.347 · Cited by 59
Evaluating and Mitigating Bias in Image Classifiers: A Causal Perspective Using Counterfactuals
Dash, S., Balasubramanian, V. N., & Sharma, A. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00393 · Cited by 59
DLIME: A Deterministic Local Interpretable Model - Agnostic Explanations Approach for Computer-Aided Diagnosis SystemsPreprint
Zafar, M. R., & Khan, N. (2023) · doi:10.32920/22734359 · Cited by 48
Optimism in the Face of Adversity: Understanding and Improving Deep Learning Through Adversarial Robustness
Ortiz-Jiménez, G., Modas, A., Moosavi-Dezfooli, S.-M., & Frossard, P. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3050042 · Cited by 48
Explain and Predict, and then Predict Again
Zhang, Z., Rudra, K., & Anand, A. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441758 · Cited by 60

Authors: Ross, A. S., Hughes, M. C., & Doshi-Velez, F.
Affiliations: Harvard University (US)
Harvard University
ORCID: Ross, Andrew Slavin
Hughes, Michael C.
Venue: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence
Publisher: International Joint Conferences on Artificial Intelligence Organization
Published: 2017-08
Pages: 2662-2670
DOI: 10.24963/ijcai.2017/371
OpenAlex: W2595017202
Cited by: 515
References: 31
Citations by year: 2026: 34 · 2025: 46 · 2024: 58 · 2023: 73 · 2022: 64 · 2021: 93 · 2020: 67 · 2019: 43 · 2018: 31 · 2017: 5
FWCI: 18.21
Type: conference paper
Language: en
Open access: bronze
doi.org
Locations: International Joint Conference on Artificial Intelligence (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: Defense Advanced Research Projects Agency (W911NF-16-1-0561)
Air Force Office of Scientific Research (FA9550-17, FA9550-, FA9550-17-1, -16-1-)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
Keywords: explainable AI, gradient-based explanation, decision boundaries, neural network interpretability, perturbation method, lime

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks