The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Explainability fact sheets: a framework for systematic assessment of explainable approaches
Open accessAbstract
Explanations in Machine Learning come in many forms, but a consensus regarding their desired properties is yet to emerge. In this paper we introduce a taxonomy and a set of descriptors that can be used to characterise and systematically assess explainable systems along five key dimensions: functional, operational, usability, safety and validation. In order to design a comprehensive and representative taxonomy and associated descriptors we surveyed the eXplainable Artificial Intelligence literature, extracting the criteria and desiderata that other authors have proposed or implicitly used in their research. The survey includes papers introducing new explainability algorithms to see what criteria are used to guide their development and how these algorithms are evaluated, as well as papers proposing such criteria from both computer science and social science perspectives. This novel framework allows to systematically compare and contrast explainability approaches, not just to better understand their capabilities but also to identify discrepancies between their theoretical qualities and properties of their implementations. We developed an operationalisation of the framework in the form of Explainability Fact Sheets, which enable researchers and practitioners alike to quickly grasp capabilities and limitations of a particular explainable method. When used as a Work Sheet, our taxonomy can guide the development of new explainability approaches by aiding in their critical evaluation along the five proposed dimensions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731
Grice, H. P. (1975) · Speech Acts · doi:10.1163/9789004368811_003 · Cited by 23,221
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469
Goldstein, A., Kapelner, A., Bleich, J., & Pitkin, E. (2015) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2014.907095 · Cited by 1,861
The Book of Why: The New Science of Cause and Effect
Goldberg, L. R. (2019) · Notices of the American Mathematical Society · doi:10.1090/noti1912 · Cited by 1,655
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514
Samarati, P., & Sweeney, L. (2018) · doi:10.1184/r1/6625469 · Cited by 1,326
The misunderstood limits of folk science: An illusion of explanatory depth
Rozenblit, L., & Keil, F. (2002) · Cognitive Science · doi:10.1207/s15516709cog2605_1 · Cited by 1,032
Bender, E. M., & Friedman, B. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00041 · Cited by 854
Model-Agnostic Interpretability of Machine Learning
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1606.05386 · Cited by 699
Extracting Tree-Structured Representations of Trained Networks
Craven, M. W., & SHAVLIK, J. W. (1995) · neural information processing systems · Cited by 602
Principles of Explanatory Debugging to Personalize Interactive Machine Learning
Kulesza, T., Burnett, M., Wong, W.-K., & Stumpf, S. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701399 · Cited by 576
Explanation, imagination, and confidence in judgment.
Koehler, D. J. (1991) · Psychological Bulletin · doi:10.1037//0033-2909.110.3.499 · Cited by 561
(2016) · neural information processing systems · Cited by 556
Current Approaches to Change Blindness
Simons, D. J. (2000) · Visual Cognition · doi:10.1080/135062800394658 · Cited by 512
A "nutrition label" for privacy
Kelley, P. G., Bresee, J., Cranor, L. F., & Reeder, R. W. (2009) · Proceedings of the 5th Symposium on Usable Privacy and Security · doi:10.1145/1572532.1572538 · Cited by 475
FactSheets: Increasing trust in AI services through supplier's declarations of conformity
Arnold, M., Bellamy, R. K. E., Hind, M., Houde, S., Mehta, S., Mojsilović, A., Nair, R., Ramamurthy, K. N., Olteanu, A., Piorkowski, D., et al. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2942288 · Cited by 434
Interacting with Predictions: Visual Inspection of Black-box Machine Learning Models
Krause, J., Perer, A., & Ng, K. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858529 · Cited by 390
Too much, too little, or just right? Ways explanations impact end users' mental models
Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., & Wong, W.-K. (2013) · 2013 IEEE Symposium on Visual Languages and Human Centric Computing · doi:10.1109/vlhcc.2013.6645235 · Cited by 388
Simplicity and probability in causal explanation
LOMBROZO, T. (2007) · Cognitive Psychology · doi:10.1016/j.cogpsych.2006.09.006 · Cited by 384
The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype ClassificationPreprint
Kim, B., Rudin, C., & Shah, J. A. (2014) · Neural Information Processing Systems · doi:10.48550/arxiv.1503.01161 · Cited by 346
FACE: Feasible and Actionable Counterfactual Explanations
Poyiadzi, R., Sokol, K., Santos-Rodriguez, R., De Bie, T., & Flach, P. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375850 · Cited by 317
Self-citationThe IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems
Chatila, R., & Havens, J. C. (2019) · Intelligent Systems, Control and Automation: Science and Engineering · doi:10.1007/978-3-030-12524-0_2 · Cited by 307
Interpretable Predictions of Tree-based Ensembles via Actionable Feature Tweaking
Tolomei, G., Silvestri, F., Haines, A., & Lalmas, M. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098039 · Cited by 276
The challenge of crafting intelligible intelligence
Weld, D. S., & Bansal, G. (2019) · Communications of the ACM · doi:10.1145/3282486 · Cited by 253 · cited 4×
“Adjusting an explanation to users requires the explainability technique to model their background knowledge and mental model [48, 55].”
BackgroundMethodInfluentialBringing Transparency Design into Practice
Eiband, M., Schneider, H., Bilandzic, M., Fazekas-Con, J., Haug, M., & Hussmann, H. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172961 · Cited by 250
Tell me more?: the effects of mental model soundness on personalizing an intelligent agent
Kulesza, T., Stumpf, S., Burnett, M., & Kwan, I. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207678 · Cited by 225
Assumption-Based Argumentation
Dung, P. M., Kowalski, R. A., & Toni, F. (2009) · Argumentation in Artificial Intelligence · doi:10.1007/978-0-387-98197-0_10 · Cited by 222
The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations
Laugel, T., Lesot, M.-J., Marsala, C., Renard, X., & Detyniecki, M. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/388 · Cited by 194
Perturbation-Based Explanations of Prediction Models
Robnik-Šikonja, M., & Bohanec, M. (2018) · Human–Computer Interaction Series · doi:10.1007/978-3-319-90403-0_9 · Cited by 171
Stakeholders in Explainable AI
Preece, A., Harborne, D., Braines, D., Tomsett, R., & Chakraborty, S. (2018) · arXiv · doi:10.48550/arxiv.1810.00184 · Cited by 117
Defining Locality for Surrogates in Post-hoc InterpretablityPreprint
Laugel, T., Renard, X., Lesot, M., Marsala, C., & Detyniecki, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.07498 · Cited by 109
A Nutritional Label for Rankings
Yang, K., Stoyanovich, J., Asudeh, A., Howe, B., Jagadish, H., & Miklau, G. (2018) · Proceedings of the 2018 International Conference on Management of Data · doi:10.1145/3183713.3193568 · Cited by 103 · cited 2×
“• quickly grasp properties of a given method to choose an appropriate approach for a desired use case given the unified design of the Fact Sheets (as with food nutrition labels [56] the users know what to expect and how to navigate them).”
MethodResult(2018) · Human-computer interaction series · doi:10.1007/978-3-319-90403-0 · Cited by 98
Explanation facilities and interactive systems
Johnson, H., & Johnson, P. (1993) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/169891.169951 · Cited by 78
Using Visual Analytics to Interpret Predictive Machine Learning ModelsPreprint
Krause, J., Perer, A., & Bertini, E. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.05685 · Cited by 70
"Why Should You Trust My Explanation?" Understanding Uncertainty in LIME Explanations
Zhang, Y., Song, K., Sun, Y., Tan, S., & Udell, M. (2019) · arXiv · doi:10.48550/arxiv.1904.12991 · Cited by 68
Dialogical Models of Explanation.
Walton, D. (2007) · Cited by 35
Interpretable & Explorable Approximations of Black Box Models
Lakkaraju, H., Kamar, E., Caruana, R., & Leskovec, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01154 · Cited by 23
Background Check: A General Technique to Build More Reliable and Versatile Classifiers
Perelló-Nieto, M., Filho, T. M. S., Kull, M., & Flach, P. A. (2016) · Proceedings · doi:10.1109/icdm.2016.0150 · Cited by 14
Self-citation
Research works printed but not linked: 15
- Biran Or, 2014, Proceedings of the AutoML workshop at ICML, 2014
- Dimitri Bohlender and Maximilian A Köhl. 2019. Towards a Characterization of Explainable Systems. arXiv preprint arXiv:1902.03096 (2019). Dimitri Bohlender and Maximilian A Köhl. 2019. Towards a Characterization of Explainable Systems. arXiv preprint arXiv:1902.03096 (2019). · arXiv
- Hall Mark, 2019, Proceedings of the IJCAI Workshop on eXplainable Artificial Intelligence (XAI
- Herman Bernease, 2017, Interpretable ML Symposium at the 31st Conference on Neural Information Processing Systems (NIPS 2017
- Sarah Holland Ahmed Hosny Sarah Newman Joshua Joseph and Kasia Chmielinski. 2018. The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards. arXiv preprint arXiv:1805.03677 (2018). Sarah Holland Ahmed Hosny Sarah Newman Joshua Joseph and Kasia Chmielinski. 2018. The Dataset Nutrition Label: A Framework To Drive Higher Data Quality Standards. arXiv preprint arXiv:1805.03677 (2018). · arXiv
- Koh Pang Wei, 2017, Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research), Doina Precup and Yee Whye Teh (Eds.), 70
- Krause Josua, 2016, 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016)
- Lakkaraju Himabindu, 2017, 4th Workshop on Fairness, Accountability, and Transparency in Machine Learning (FAT/ML 2017) at the 23rd SIGKDD conference on Knowledge Discovery and Data Mining (KDD 2017
- Laugel Thibault, 2018, 3rd Workshop on Human Interpretability in Machine Learning (WHI 2018) at the 35th International Conference on Machine Learning (ICML 2018
- Miller Tim, 2017, Explainable AI: Beware of Inmates Running the Asylum. In IJCAI-17 Workshop on Explainable AI (XAI). 36
- Ribeiro Marco Tulio, 2016, 1st Workshop on Human Interpretability in Machine Learning (WHI 2016) at the 33rd International Conference on Machine Learning (ICML 2016
- Robnik-Šikonja Marko, Human and Machine Learning
- Tomsett Richard, 2018, 3rd Workshop on Human Interpretability in Machine Learning (WHI 2018) at the 35th International Conference on Machine Learning (ICML 2018
- Sandra Wachter Brent Mittelstadt and Chris Russell. 2018. Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR. Harvard Journal of Law & Technology 31 (2) (2018). Sandra Wachter Brent Mittelstadt and Chris Russell. 2018. Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR. Harvard Journal of Law & Technology 31 (2) (2018). · DOI · Paper page
- Zhang Yujia, 2019, AI for Social Good Workshop at the 36th International Conference on Machine Learning (ICML 2019
Cited by
Shin, D. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102551 · Cited by 1,548
Trustworthy Artificial Intelligence: A Review
Kaur, D., Uslu, S., Rittichier, K. J., & Durresi, A. (2023) · ACM Computing Surveys · doi:10.1145/3491209 · Cited by 709
Langer, M., Oster, D., Speith, T., Hermanns, H., Kästner, L., Schmidt, E., Sesing, A., & Baum, K. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103473 · Cited by 652
Nauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schlötterer, J., van Keulen, M., & Seifert, C. (2023) · ACM Computing Surveys · doi:10.1145/3583558 · Cited by 590
A Review of Taxonomies of Explainable Artificial Intelligence (XAI) Methods
Speith, T. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3534639 · Cited by 337
Metadata
- Authors
- Sokol, K., & Flach, P.
- Venue
- Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2020-01-27
- Pages
- 56-67
- OpenAlex
- W4288083797
- Cited by
- 264
- References
- 46
- FWCI
- 14.01
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.