Explainable machine learning in deployment
Open accessThe DOI matches its Crossref record; 82 of the 83 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Explainable machine learning offers the potential to provide stakeholders with insights into model behavior by using various methods such as feature importance scores, counterfactual explanations, or influential training data. Yet there is little understanding of how organizations use these methods in practice. This study explores how organizations view and use explainability for stakeholder consumption. We find that, currently, the majority of deployments are not for end users affected by the model but rather for machine learning engineers, who use explainability to debug the model itself. There is thus a gap between explainability in practice and the goal of transparency, since explanations primarily serve internal stakeholders rather than external ones. Our study synthesizes the limitations of current explainability techniques that hamper their use for end users. To facilitate end user interaction, we develop a framework for establishing clear goals for explainability. We end by discussing concerns raised regarding explainability.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Though explanations can assist some specific concerns, like model engineering [7], explanations or other approaches focused on how the model itself works or has reached a particular outcome may miss much of what is important [75].”
From Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems (2021) · cites this paper as [7]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“However, most existing explainable AI (XAI) methods [10,21,43] target math-savy AI creators [1,22] to aid them in AI model debugging and monitoring [6].”
From Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust (2023) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“Note that we chose to provide SHAP explanations for one black-box model because it emulates how black-box models are most commonly implemented in-practice [10,36].”
From It’s Just Not That Simple: An Empirical Study of the Accuracy-Explainability Trade-off in Machine Learning for Public Policy (2022) · cites this paper as [10]
In the citing paper
“As many researchers have articulated, safe and effective deployment of AI/ML systems in practice requires a robust understanding of different stakeholders' needs and goals for explainability [8, 32]; support for traceability and auditing [71]; and strategies to address concerns within existing organizational structures and dynamics [8, 31, 32, 57, 71].”
From Interactive Model Cards: A Human-Centered Approach to Model Documentation (2022) · cites this paper as [8]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Gauriat, C.-M., Pencolé, Y., Ribot, P., & Brouillet, G. (2024) · HAL (Le Centre pour la Communication Scientifique Directe) · doi:10.4230/oasics.dx.2024.27 · Cited by 12,586
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · cited 2×
“In fact, the power of ML tools is often presented as a trade-off between accuracy and explainability, with simpler more interpretable models often performing worse on accuracy measures [49].”
Background17. A Value for n-Person Games
Shapley, L. S. (1953) · Contributions to the Theory of Games (AM-28), Volume II · doi:10.1515/9781400881970-018 · Cited by 6,950
Towards Deep Learning Models Resistant to Adversarial Attacks
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2017) · arXiv · doi:10.48550/arxiv.1706.06083 · Cited by 6,602
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Statistics and Causal Inference
Holland, P. W. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478354 · Cited by 5,293 · cited 1×
“While the term counterfactual has awell understoodmeaning in the causality literature [28, 45], counterfactual explanations forML”
BackgroundShapley, L. S. (1988) · The Shapley Value · doi:10.1017/cbo9780511528446.003 · Cited by 4,954
CAUSALITY: MODELS, REASONING AND INFERENCE
Hitchcock, C. (2001) · The Philosophical Review · doi:10.1215/00318108-110-4-639 · Cited by 4,741
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087
Self-citationTowards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
Learning Important Features Through Propagating Activation Differences
Shrikumar, A., Greenside, P., & Kundaje, A. (2017) · arXiv · doi:10.48550/arxiv.1704.02685 · Cited by 2,761
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Clinically applicable deep learning for diagnosis and referral in retinal disease
De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., Askham, H., Glorot, X., O’Donoghue, B., Visentin, D., et al. (2018) · Nature Medicine · doi:10.1038/s41591-018-0107-6 · Cited by 2,639
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · cited 2×
“Once the context and utility of the explanation are stated, understand what the stakeholder wants to do with the explanation [24].”
BackgroundMethodExplaining prediction models and individual predictions with feature contributions
Štrumbelj, E., & Kononenko, I. (2014) · Knowledge and Information Systems · doi:10.1007/s10115-013-0679-x · Cited by 2,286 · cited 1×
“These explanation functionals roughly fall into two categories: perturbation-based techniques [9, 16, 23, 35, 48, 58] and gradient-based techniques [8, 41, 53, 55, 56, 59].”
BackgroundDetection of Influential Observation in Linear Regression
Cook, R. D. (2000) · Technometrics · doi:10.2307/1271434 · Cited by 2,269
Detection of Influential Observation in Linear Regression
Cook, R. D. (1977) · Technometrics · doi:10.1080/00401706.1977.10489493 · Cited by 2,268
Explainable machine-learning predictions for the prevention of hypoxaemia during surgery
Lundberg, S. M., Nair, B., Vavilala, M. S., Horibe, M., Eisses, M. J., Adams, T., Liston, D. E., Low, D. K.-W., Newman, S.-F., Kim, J., et al. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0304-0 · Cited by 1,967
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · cited 3×
“In other words, they hope to identify the influential training data points [32] to understand which past drivers had the greatest influence on the prediction for the observed driver.”
BackgroundInfluentialSmoothGrad: removing noise by adding noise
Smilkov, D., Thorat, N., Kim, B., Viégas, F., & Wattenberg, M. (2017) · arXiv · doi:10.48550/arxiv.1706.03825 · Cited by 1,467 · cited 1×
“based techniques [7, 41, 57, 59, 60, 62].”
The State of Applied Econometrics: Causality and Policy Evaluation
Athey, S., & Imbens, G. W. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.3 · Cited by 1,397
Explainable AI: Interpreting, Explaining and Visualizing Deep Learning
(2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6 · Cited by 1,377
Explaining nonlinear classification decisions with deep taylor decomposition
Montavon, G., Lapuschkin, S., Binder, A., Samek, W., & Müller, K.-R. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.11.008 · Cited by 1,357
Concrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314 · cited 1×
“Ideally, they would like to be alerted when there is a significant drift relative to the training distribution [6, 47].”
BackgroundKim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., & Sayres, R. (2017) · arXiv · doi:10.48550/arxiv.1711.11279 · Cited by 1,240
Board of Governors of the Federal Reserve System
(2001) · Choice Reviews Online · doi:10.5860/choice.38sup-465 · Cited by 1,130
Interpretable Explanations of Black Boxes by Meaningful Perturbation
Fong, R. C., & Vedaldi, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.371 · Cited by 1,097 · cited 2×
“also a type of feature-level explanation that denote the importance of a region or collection of features [4, 22].”
BackgroundCounterfactual explanations without opening the black box: automated decisions and the GDPR
Wachter, S., Mittelstadt, B., & Russell, C. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3063289 · Cited by 1,061
Aas, K., Jullum, M., & Løland, A. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103502 · Cited by 1,023
Adversarial Examples Are Not Bugs, They Are Features
Ilyas, A., Santurkar, S., Tsipras, D., Engstrom, L., Tran, B., & Madry, A. (2019) · arXiv · doi:10.48550/arxiv.1905.02175 · Cited by 971
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., Dafoe, A., Scharre, P., Zeitzoff, T., Filar, B., et al. (2018) · Apollo - University of Cambridge Repository · doi:10.17863/cam.22520 · Cited by 849
Self-citationThe Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., Dafoe, A., Scharre, P., Zeitzoff, T., Filar, B., et al. (2018) · arXiv · doi:10.48550/arxiv.1802.07228 · Cited by 849
Self-citationStealing Machine Learning Models via Prediction APIs
Tramèr, F., Zhang, F., Juels, A., Reiter, M. K., & Ristenpart, T. (2016) · arXiv · doi:10.48550/arxiv.1609.02943 · Cited by 796
Mittelstadt, B., Russell, C., & Wachter, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287574 · Cited by 794 · cited 1×
“Contrastive explanations that highlight contextually relevant information to the model output are most similar to human explanations[40]; however, in their current form, finding the relevant set of plausible counterfactual point is no clear.”
BackgroundImproving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777 · cited 1×
“[28], we study how industry practitioners look at and deploy explainable ML.”
BackgroundInterpretation of Neural Networks Is Fragile
Ghorbani, A., Abid, A., & Zou, J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33013681 · Cited by 765
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · cited 1×
“• Static Consumption: Will the explanation be used as a oneoff sanity check for some stakeholders or shown to other stakeholders as reasoning for a particular prediction [48]? • Dynamic Model Updates: Will the explanation be used to garner feedback from the stakeholder as to how themodel ought to be updated to better…”
BackgroundManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Vaughan, J. W., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1802.07810 · Cited by 711
Lepri, B., Oliver, N., Letouzé, E., Pentland, A., & Vinck, P. (2018) · Philosophy & Technology · doi:10.1007/s13347-017-0279-x · Cited by 709
Top-Down Neural Attention by Excitation Backprop
Zhang, J., Bargal, S. A., Lin, Z., Brandt, J., Shen, X., & Sclaroff, S. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-017-1059-x · Cited by 680
Kleinberg, J., Ludwig, J., Mullainathan, S., & Obermeyer, Z. (2015) · American Economic Review · doi:10.1257/aer.p20151023 · Cited by 584 · cited 1×
“The relationship between causality and explainability in MLrelatestoabroaderdichotomy.IndistinguishingMLtechniquesfrom those in econometrics or statistics, a key component is prediction vs. inference [10, 11, 33]. ML focuses on prediction, such that understandingtheunderlyingprocessisgenerallyconsideredlessimportant t…”
BackgroundBeyond prediction: using big data for policy problems
Athey, S. (2017) · Science · doi:10.1126/science.aal4321 · Cited by 575 · cited 1×
“The relationship between causality and explainability in MLrelatestoabroaderdichotomy.IndistinguishingMLtechniquesfrom those in econometrics or statistics, a key component is prediction vs. inference [10, 11, 33]. ML focuses on prediction, such that understandingtheunderlyingprocessisgenerallyconsideredlessimportant t…”
Background(2016) · neural information processing systems · Cited by 556
Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations
Ross, A. S., Hughes, M. C., & Doshi-Velez, F. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/371 · Cited by 515 · cited 1×
“[51] attempt to develop a technique for dynamic explanations, wherein the human can guide the model towards learning the correct explanation.”
BackgroundThe What-If Tool: Interactive Probing of Machine Learning Models
Wexler, J., Pushkarna, M., Bolukbasi, T., Wattenberg, M., Viegas, F., & Wilson, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934619 · Cited by 505
Actionable Recourse in Linear Classification
Ustun, B., Spangher, A., & Liu, Y. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287566 · Cited by 501
The Intuitive Appeal of Explainable Machines
Selbst, A. D., & Barocas, S. (2018) · doi:10.2139/ssrn.3126971 · Cited by 392
Robustness May Be at Odds with Accuracy
Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., & Mądry, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.12152 · Cited by 327
Hooker, G., Mentch, L., & Zhou, S. (2021) · Statistics and Computing · doi:10.1007/s11222-021-10057-z · Cited by 264
Suzor, N. P., West, S. M., Quodling, A., & York, J. C. (2019) · International journal of communication · Cited by 257
Linking Trust to Trustworthiness
O’Neill, O. (2018) · International Journal of Philosophical Studies · doi:10.1080/09672559.2018.1454637 · Cited by 233
Sharma, S., Henderson, J., & Ghosh, J. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375812 · Cited by 226
Self-citationL-Shapley and C-Shapley: Efficient Model Interpretation for Structured DataPreprint
Chen, J., Song, L., Wainwright, M. J., & Jordan, M. I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.02610 · Cited by 221
Transparency: Motivations and Challenges
Weller, A. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6_2 · Cited by 198
Self-citationExplanations can be manipulated and geometry is to blame
Dombrowski, A.-K., Alber, M., Anders, C. J., Ackermann, M., Müller, K.-R., & Kessel, P. (2019) · arXiv · doi:10.48550/arxiv.1906.07983 · Cited by 171 · cited 1×
“tion between adversarial robustness and model interpretations [18, 21, 23, 59, 69].”
BackgroundOn the Connection Between Adversarial Robustness and Saliency Map InterpretabilityPreprint
Etmann, C., Lunz, S., Maaß, P., & Schönlieb, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.04172 · Cited by 169
Model Reconstruction from Model Explanations
Milli, S., Schmidt, L., Dragan, A. D., & Hardt, M. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287562 · Cited by 141
Towards Algorithmic Experience: Initial Efforts for Social Media Contexts
Alvarado, O., & Waern, A. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173860 · Cited by 127
Stakeholders in Explainable AI
Preece, A., Harborne, D., Braines, D., Tomsett, R., & Chakraborty, S. (2018) · arXiv · doi:10.48550/arxiv.1810.00184 · Cited by 117 · cited 1×
“[49] describe how stakeholders have different needs for explainability.”
BackgroundOn the Privacy Risks of Model Explanations
Shokri, R., Strobel, M., & Zick, Y. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462533 · Cited by 111
Please Stop Permuting Features: An Explanation and Alternatives.Preprint
Hooker, G., & Mentch, L. (2019) · arXiv (Cornell University) · Cited by 102
Deep Learning in FinancePreprint
Heaton, J. B., Polson, N. G., & Witte, J. H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.06561 · Cited by 87
"Why Should You Trust My Explanation?" Understanding Uncertainty in LIME Explanations
Zhang, Y., Song, K., Sun, Y., Tan, S., & Udell, M. (2019) · arXiv · doi:10.48550/arxiv.1904.12991 · Cited by 68
Bargal, S. A., Zunino, A., Kim, D., Zhang, J., Murino, V., & Sclaroff, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00156 · Cited by 62
On the (In)fidelity and Sensitivity for Explanations
Yeh, C., Hsieh, C.-Y., Suggala, A. S., Inouye, D. I., & Ravikumar, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09392 · Cited by 60
(2018) · ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) · Cited by 57
Self-citationHow to Explain Individual Classification Decisions
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1859912 · Cited by 54 · cited 1×
“Also known as feature-level interpretations, feature attributions, or saliency maps, this method is by far the most widely used and most well-studied explainability technique [9, 24].”
MethodUnderstanding Impacts of High-Order Loss Approximations and Features in Deep Learning InterpretationPreprint
Singla, S., Wallace, E., Feng, S., & Feizi, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00407 · Cited by 54
Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation
Ancona, M., Öztireli, C., & Groß, M. H. (2019) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000342737 · Cited by 42
Hooker, G., Mentch, L., & Zhou, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.03151 · Cited by 37
Data Squashing: Constructing Summary Data Sets
DuMouchel, W. (2002) · Massive computing · doi:10.1007/978-1-4615-0005-6_16 · Cited by 33
Neural Network Attributions: A Causal Perspective
Chattopadhyay, A., Manupriya, P., Sarkar, A., & Balasubramanian, V. N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.02302 · Cited by 31
Sharma, S., Henderson, J., & Ghosh, J. (2019) · arXiv · doi:10.48550/arxiv.1905.07857 · Cited by 28
Self-citationPrivacy Risks of Explaining Machine Learning Models.Preprint
Shokri, R., Strobel, M., & Zick, Y. (2019) · arXiv (Cornell University) · Cited by 21
Improving Simple Models with Confidence Profiles
Dhurandhar, A., Shanmugam, K., Luss, R., & Olsen, P. A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.07506 · Cited by 17
How Sensitive are Sensitivity-Based Explanations?
Yeh, C., Hsieh, C.-Y., Suggala, A. S., Inouye, D. I., & Ravikumar, P. (2019) · Cited by 12
Towards Aggregating Weighted Feature AttributionsPreprint
Bhatt, U., Ravikumar, P., & Moura, J. M. F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.10040 · Cited by 11
Self-citationAutomatic Model Monitoring for Data StreamsPreprint
Pinto, F., Sampaio, M. O. P., & Bizarro, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.04240 · Cited by 7
Interpretable and Differentially Private Predictions
Harder, F., Bauer, M., & Park, M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5827 · Cited by 6
Word Embedding based Edit DistancePreprint
Niu, Y., Qiao, C., Li, H., & Huang, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.10752 · Cited by 4
Shrikumar, A., Prakash, E., & Kundaje, A. B. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/457606 · Cited by 4
Research works printed but not linked: 3
- European Parliament and Council of European Union. 2018 . European Union General Data Protection Regulation , Articles 13 - 15 . http://www.privacy-regulation.eu/en/13.htm (2018). European Parliament and Council of European Union. 2018. European Union General Data Protection Regulation, Articles 13-15. http://www.privacy-regulation.eu/en/13.htm (2018).
- 10.5555/3305890.3306006 · doi:10.5555/3305890.3306006
- 10.5555/3305890.3306024 · doi:10.5555/3305890.3306024
Cited by
Explainable AI (XAI): Core Ideas, Techniques, and Solutions
Dwivedi, R., Dave, D., Naik, H., Singhal, S., Omer, R., Patel, P., Qian, B., Wen, Z., Shah, T., Morgan, G., et al. (2023) · ACM Computing Surveys · doi:10.1145/3561048 · Cited by 1,386
Deep Neural Networks and Tabular Data: A Survey
Borisov, V., Leemann, T., Seßler, K., Haug, J., Pawelczyk, M., & Kasneci, G. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3229161 · Cited by 1,025
Interpretable machine learning: Fundamental principles and 10 grand challenges
Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., & Zhong, C. (2022) · Statistics Surveys · doi:10.1214/21-ss133 · Cited by 958
Interpretable machine learning
Parliamentary Office of Science and Technology, & Christie, L. (2020) · doi:10.58248/pn633 · Cited by 952
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757
“s, including whether the explanation improves users’ trust on the AI [34, 68] or enables users to simulate the model predictions [13, 55], or assisting developers to debug and train a better ML model [7,30]. , Vol. 1, No. 1, Article . Publication date: July 2020. 4 G. Bansal et al. Domain Task Performance Metric Human…”
Background
Metadata
- Authors
- Bhatt, U., Xiang, A., Sharma, S., Weller, A., Taly, A., Jia, Y., Ghosh, J., Puri, R., Moura, J. M. F., & Eckersley, P.
- Affiliations
- University of Cambridge (GB)
- Partnership on AI
- The University of Texas at Austin (US)
- Fiddler Labs
- Baidu (China) (CN)
- IBM (United States) (US)
- Carnegie Mellon University (US)
- ORCID
- Venue
- Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2020-01-27
- Pages
- 648-657
- OpenAlex
- W2973193541
- Semantic Scholar
- 4490ffac416692ab827c5a30e5f3a4b4fd6be949
- Cited by
- 633
- References
- 77
- Citations by year
- 2026: 46 · 2025: 93 · 2024: 92 · 2023: 135 · 2022: 129 · 2021: 109 · 2020: 27
- FWCI
- 38.19
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- arXiv (Cornell University)
- Funders
- David MacKay Newton research fellowship at Darwin College
- The Alan Turing Institute (EPSRC EP/N510129/1, TU/B/000074)
- DeepMind via the Leverhulme Centre for the Future of Intelligence (CFI)
- Partnership on AI research fellowship
- Leverhulme Trust via CFI
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- explainable machine learning, machine learning deployment, model transparency, feature importance, counterfactual explanations, model debugging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.