The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
PALM: Machine Learning Explanations For Iterative Debugging
Abstract
When a Deep Neural Network makes a misprediction, it can be challenging for a developer to understand why. While there are many models for interpretability in terms of predictive features, it may be more natural to isolate a small set of training examples that have the greatest influence on the prediction. However, it is often the case that every training example contributes to a prediction in some way but with varying degrees of responsibility. We present Partition Aware Local Model (PALM), which is a tool that learns and summarizes this responsibility structure to aide machine learning debugging. PALM approximates a complex model (e.g., a deep neural network) using a two-part surrogate model: a meta-model that partitions the training data, and a set of sub-models that approximate the patterns within each partition. These sub-models can be arbitrarily complex to capture intricate local patterns. However, the meta-model is constrained to be a decision tree. This way the user can examine the structure of the meta-model, determine whether the rules match intuition, and link problematic test examples to responsible training data efficiently. Queries to PALM are nearly 30x faster than nearest neighbor queries for identifying relevant data, which is a key property for interactive applications.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In fact, removing the influence of outlier training samples from a model will lead to higher model performance and robustness [16].”
From Machine Unlearning: A Survey (2024) · cites this paper as [16]
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · cited 2×
“Furthermore, highly expressive deep models such as deep neural networks, which in principle can learn any deterministic function, are susceptible to adversarial examples (i.e., imperceptible perturbations to the features that cause a change in prediction)[15].”
BackgroundRationalizing Neural Predictions
Lei, T., Barzilay, R., & Jaakkola, T. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1011 · Cited by 777 · cited 4×
“[13, 14, 16].”
BackgroundInfluentialProvenance in Databases: Why, How, and Where
Cheney, J., Chiticariu, L., & Tan, W.-C. (2009) · Foundations and Trends in Databases · doi:10.1561/1900000006 · Cited by 547 · cited 1×
“This notion is similar to responsibility in the data lineage and explanation literature [4, 17], in the context of black-box machine learning models as opposed to database queries with well defined semantics.”
ResultThe MADlib analytics library: or MAD skills, the SQL
Hellerstein, J. M., Ré, C., Schoppmann, F., Wang, D. Z., Fratkin, E., Gorajek, A., Ng, K. S., Welton, C., Feng, X., Li, K., et al. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2367502.2367510 · Cited by 373
Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention
Kim, J., & Canny, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.320 · Cited by 366 · cited 1×
“Another more recent approach in computer vision is to use attention models, enforce that the model focuses on certain features, for ex-plainability [9].”
BackgroundMLbase: A Distributed Machine-learning System
Kraska, T., Talwalkar, A., & Duchi, J. C. (2013) · Cited by 306
ActiveClean: interactive data cleaning for statistical modeling
Krishnan, S., Wang, J., Wu, E., Franklin, M. J., & Goldberg, K. (2016) · Proceedings of the VLDB Endowment · doi:10.14778/2994509.2994514 · Cited by 264 · cited 1×
“Explaining a prediction exactly in terms of features is highly useful for an end-user, but developers also need to be able to trace modeling errors to training data [12].”
BackgroundSelf-citationScorpion: explaining away outliers in aggregate queries
Wu, E., & Madden, S. (2013) · Proceedings of the VLDB Endowment · doi:10.14778/2536354.2536356 · Cited by 240 · cited 1×
“This notion is similar to responsibility in the data lineage and explanation literature [4, 17], in the context of black-box machine learning models as opposed to database queries with well defined semantics.”
ResultSelf-citationMacroBase: Prioritizing Attention in Fast Data
Bailis, P. D., Gan, E., Madden, S. R., Narayanan, D., Rong, K., & Suri, S. (2017) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3035918.3035928 · Cited by 135 · cited 1×
“We believe that coupling explanations with anomaly detection may be a viable next step, as in the MacroBase project [3].”
Method
Research works printed but not linked: 3
- Andrew Crotty , Alex Galakatos , and Tim Kraska . 2014 . Tupleware: Distributed Machine Learning on Small Clusters . In IEEE Data Eng. Bull . http://sites.computer.org/debull/A14sept/p63.pdf Andrew Crotty, Alex Galakatos, and Tim Kraska. 2014. Tupleware: Distributed Machine Learning on Small Clusters. In IEEE Data Eng. Bull. http://sites.computer.org/debull/A14sept/p63.pdf
- Roy Fox* Sanjay Krishnan* Ken Goldberg and Ion Stoica. 2017. Multi-Layer Deep Option Discovery. In Under Review ICML. Roy Fox* Sanjay Krishnan* Ken Goldberg and Ion Stoica. 2017. Multi-Layer Deep Option Discovery. In Under Review ICML.
- Tameru Hailesilassie . 2016. Rule Extraction Algorithm for Deep Neural Networks: A Review. arXiv preprint arXiv:1610.05267 ( 2016 ). Tameru Hailesilassie. 2016. Rule Extraction Algorithm for Deep Neural Networks: A Review. arXiv preprint arXiv:1610.05267 (2016). · arXiv
Cited by
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Machine Learning Testing: Survey, Landscapes and Horizons
Zhang, J. M., Harman, M., Ma, L., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2962027 · Cited by 897
Interpretable Machine Learning – A Brief History, State-of-the-Art and Challenges
Molnar, C., Casalicchio, G., & Bischl, B. (2020) · Communications in Computer and Information Science · doi:10.1007/978-3-030-65965-3_28 · Cited by 677
“Many IML methods are surrogate model approaches [89,75,72,95,34,10,18,61] and differ, e.”
MethodA Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629
Metadata
- Authors
- Krishnan, S., & Wu, E.
- Affiliations
- University of California, Berkeley (US)
- Columbia University (US)
- Venue
- Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics
- Publisher
- ACM
- Published
- 2017-05-14
- Pages
- 1-6
- OpenAlex
- W2614087582
- Semantic Scholar
- 6315d7f2619d9ec05846765d1283dee28bce9bad
- Cited by
- 81
- References
- 13
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 15 · 2023: 5 · 2022: 10 · 2021: 15 · 2020: 11 · 2019: 11 · 2018: 4 · 2017: 2
- FWCI
- 3.49
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Workshop on Human-In-the-Loop Data Analytics (HILDA) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- interpretability, training data influence, surrogate models, deep neural network
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: PALM: Machine Learning Explanations For Iterative Debugging. https://app.cochat.ai/papers/10.1145/3077257.3077271
- The paper
- Krishnan, S., & Wu, E. (2017). PALM: Machine Learning Explanations For Iterative Debugging. Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics. https://doi.org/10.1145/3077257.3077271
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.