The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery.
Open accessThe DOI matches its Crossref record; 24 of the 27 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Supervised machine-learning models boast remarkable predictive capabilities. But can you trust your model? Will it work in deployment? What else can it tell you about the world?
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Hence, trust in ML models is strongly related to their interpretability, which has thus become a focus of active research in the ML community 116,132 .”
From Machine learning in concrete science: applications, challenges, and best practices (2022) · cites this paper as [116]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As the field is still in its infancy, there is no consensus on the definition of the terms interpretability and explainability in this context, and recently various definitions have been proposed for these terms [9,88,110].”
From Transparency of deep neural networks for medical image analysis: A review of interpretability methods (2022) · cites this paper as [88]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Newer methodologies have the danger of becoming more complex and less interpretable, even when sophisticated interpretation technique are used11.”
From Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist. (2020) · cites this paper as [11]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Lipton (2018) [31] and Gunning et al. (2019) [32] highlight that for AI to be effectively integrated into clinical practice, it must provide clear and interpretable explanations of its outputs.”
From Leading with AI in critical care nursing: challenges, opportunities, and the human factor (2024) · cites this paper as Lipton (2018) [31]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The proposed approach to algorithmic transparency deviates from the existing body of literature on explainable artificial intelligence (xAI), where the concept of transparency focuses on the explanation of the inner workings of algorithms or the interpretability of their individual outcomes (Lipton 2018; Ribeiro et al. 2016).”
From Transparency as design publicity: explaining and justifying inscrutable algorithms (2021) · cites this paper as Lipton 2018
Experimental — accuracy not yet measured
28 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482 · cited 1×
“Note, however, that humans exhibit none of these forms of transparency.”
Background(2008) · Journal of Machine Learning Research · Cited by 33,051
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Causality: Models, Reasoning, and Inference
Pearl, J. (2009) · doi:10.1017/cbo9780511803161 · Cited by 11,653
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · cited 2×
“The CNNs were made to misclassify images that were im-perceptibly (to a human) perturbed (Szegedy et al., 2013).”
BackgroundMethodDeep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370
Dueling Network Architectures for Deep Reinforcement Learning
Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., & de Freitas, N. (2015) · arXiv · doi:10.48550/arxiv.1511.06581 · Cited by 3,300
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 1×
“A popular approach for deep neural nets, saliency maps compute gradients of predictions with respect to inputs, rendering the gradient as an image (Simonyan et al., 2013).”
BackgroundEuropean Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · cited 2×
“Few would suggest that a human could contemplate a thousand parameters at once, but most could simulate a linear model with 20 parameters.”
Background(2009) · Cited by 1,900
Research works printed but not linked: 6
- Doshi-Velez, Finale, Wallace, Byron, and Adams, Ryan. Graph-sparse lda: a topic model with structured sparsity. AAAI , 2015.
- Doshi-Velez F., 2015, Proceedings of the 29th Association for the Advancement of Artificial Intelligence (AAAI) Conference, 2575-2581
- Kim, Been, Rudin, Cynthia, and Shah, Julie A. The Bayesian Case Model: A generative approach for casebased reasoning and prototype classification. In NIPS , 2014.
- Lipton, Zachary C, Kale, David C, and Wetzel, Randall. Modeling missing data in clinical time series with rnns. In Machine Learning for Healthcare , 2016.
- Mikolov, Tomas, Sutskever, Ilya, Chen, Kai, Corrado, Greg S, and Dean, Jeff. Distributed representations of words and phrases and their compositionality. In NIPS , pp. 3111-3119, 2013.
- Ridgeway, Greg, Madigan, David, Richardson, Thomas, and O'Kane, John. Interpretable boosted na¨ ıve bayes classification. In KDD , 1998.
Cited by
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
“The opposite of black-box-ness is transparency , i.e., the search for a direct understanding of the mechanism by which a model works [5] .”
InfluentialBarredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01228-7 · Cited by 6,909
“While these deep neural networks enable superior performance, their lack of decomposability into intuitive and understandable components makes them hard to interpret [26].”
BackgroundMethodA Comprehensive Survey on Transfer Learning
Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., & He, Q. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2020.3004555 · Cited by 6,678
“Third, the interpretability of transfer learning also needs to be investigated further [216].”
Background
Metadata
- Authors
- Lipton, Z. C.
- Affiliations
- Carnegie Mellon University (US)
- Corresponding author
- Lipton, Zachary C.
- ORCID
- Lipton, Zachary C.
- Venue
- Queue
- Published
- 2018-06
- Volume
- 16
- Issue
- 3
- Pages
- 31-57
- ISSN
- 0001-0782 (ISSN-L)
- 1542-7730
- 1542-7749
- OpenAlex
- W2439568532
- Cited by
- 5,427
- References
- 27
- Citations by year
- 2026: 803 · 2025: 918 · 2024: 633 · 2023: 657 · 2022: 575 · 2021: 664 · 2020: 521 · 2019: 395 · 2018: 190
- FWCI
- 249.23
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Communications of the ACM (publishedVersion)
- Queue (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- model interpretability, explainable AI, machine learning models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery. https://app.cochat.ai/papers/10.1145/3236386.3241340
- The paper
- Lipton, Z. C. (2018). The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue. https://doi.org/10.1145/3236386.3241340
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.