Incorporating Priors with Feature Attribution on Text Classification
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Feature attribution methods, proposed recently, help users interpret the predictions of complex models.Our approach integrates feature attributions into the objective function to allow machine learning practitioners to incorporate priors in model building.To demonstrate the effectiveness our technique, we apply it to two tasks: (1) mitigating unintended bias in text classifiers by neutralizing identity terms; (2) improving classifier performance in a scarce data setting by forcing the model to focus on toxic terms.Our approach adds an L 2 distance loss between feature attributions and task-specific prior values to the objective.Our experiments show that i) a classifier trained with our technique reduces undesired model biases without a tradeoff on the original task; ii) incorporating priors helps model performance in scarce data settings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427 · cited 2×
“Lipton (2018) briefly questions the practical benefit of having model explanations from a practitioners perspective.”
BackgroundMethodExplanation in Artificial Intelligence: Insights from the Social Sciences
Miller, T. (2017) · arXiv · doi:10.48550/arxiv.1706.07269 · Cited by 5,338
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524 · cited 7×
“Another avenue is to incorporate different model attribution strategies such as DeepLRP (Bach et al., 2015) into the objective function.”
BackgroundMethodInfluentialAxiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087
Learning Important Features Through Propagating Activation Differences
Shrikumar, A., Greenside, P., & Kundaje, A. (2017) · arXiv · doi:10.48550/arxiv.1704.02685 · Cited by 2,761
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 2×
“We use AUC along with False Positive Equality Difference (FPED) and False Negative Equality Difference (FNED), which measure a proxy of Equality of Odds (Hardt et al., 2016), as in Dixon et al. (2018); Park et al. (2018).”
MethodExplaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · cited 1×
“Other sets of papers (Miller, 2019; Gilpin et al., 2018) attack model explanation papers from a philosophical perspective.”
BackgroundEuropean Union Regulations on Algorithmic Decision Making and a “Right to Explanation”
Goodman, B., & Flaxman, S. (2017) · AI Magazine · doi:10.1609/aimag.v38i3.2741 · Cited by 2,131 · cited 2×
“From healthcare to financial domains, regulatory agencies mandate entities to provide explanations for their decisions (Goodman and Flaxman, 2016).”
BackgroundMan is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · cited 3×
“It removes the necessity to initialize word embeddings with pre-debiased embeddings as proposed in Bolukbasi et al. (2016).”
BackgroundInfluentialUnderstanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · cited 1×
“One family of interpretability methods uses sensitivity of the network with respect to data points ( Koh and Liang, 2017) or features (Ribeiro et al., 2016) as a form of explanation.”
MethodSmoothGrad: removing noise by adding noise
Smilkov, D., Thorat, N., Kim, B., Viégas, F., & Wattenberg, M. (2017) · arXiv · doi:10.48550/arxiv.1706.03825 · Cited by 1,467 · cited 5×
“Smilkov et al. (2017) show that gradient based methods are susceptible to saturation and can be fooled by adversarial techniques.”
BackgroundMethodInfluentialConsistent Individualized Feature Attribution for Tree Ensembles
Lundberg, S. M., Erion, G. G., & Lee, S.-I. (2018) · arXiv · doi:10.48550/arxiv.1802.03888 · Cited by 1,430
Adversarial Examples for Evaluating Reading Comprehension Systems
Jia, R., & Liang, P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1215 · Cited by 1,410 · cited 2×
“Hence, interpretability methods tailored for text are quite sparse (Mu-drakarta et al., 2018; Jia and Liang, 2017; Mur-doch et al., 2018).”
MethodMitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387 · cited 6×
“Instead, researchers try to reduce biases indirectly by mostly adding more data (Dixon et al., 2018; Chen et al., 2018), using unbiased word vectors (Park et al., 2018), or directly optimizing for a fairness proxy with adversarial training (Madras et al., 2018; Zhang et al., 2018a).”
BackgroundMethodInfluentialSanity Checks for Saliency Maps
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294 · cited 3×
“However, the input gradients technique has been shown to be an inaccurate explanation technique (Adebayo et al., 2018).”
BackgroundMethodInfluentialKim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., & Sayres, R. (2017) · arXiv · doi:10.48550/arxiv.1711.11279 · Cited by 1,240
Rationalizing Neural Predictions
Lei, T., Barzilay, R., & Jaakkola, T. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1011 · Cited by 777 · cited 2×
“Sentence On the other hand, the amount of research focusing on explainable natural language processing (NLP) models (Li et al., 2016; Murdoch et al., 2018; Lei et al., 2016) is modest as opposed to image explanation techniques.”
BackgroundMeasuring and Mitigating Unintended Bias in Text Classification
Dixon, L., Li, J., Sorensen, J., Thain, N., & Vasserman, L. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278729 · Cited by 674 · cited 15×
“Unlike previous approaches (Park et al., 2018; Dixon et al., 2018; Madras et al., 2018), our method does not degrade classifier performance (it even improves) in terms of all reported metrics.”
BackgroundMethodInfluentialEx Machina: Personal Attacks Seen at Scale
Wulczyn, E., Thain, N., & Dixon, L. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052591 · Cited by 636
Intelligible models for classification and regression
Lou, Y., Caruana, R., & Gehrke, J. (2012) · Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2339530.2339556 · Cited by 547 · cited 1×
“To some, non-linear models such as gradient boosting trees and neural networks are less favorable due to the fact that they do not enjoy such transparent contribution of each feature and are harder to interpret (Lou et al., 2012).”
BackgroundRight for the Right Reasons: Training Differentiable Models by Constraining their Explanations
Ross, A. S., Hughes, M. C., & Doshi-Velez, F. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/371 · Cited by 515 · cited 1×
“The authors of Ross et al. (2017) propose a similar idea to our approach in that they regularize input gradients to alter the decision boundary of the model to make it more consistent with domain knowledge.”
MethodSayres, R., Taly, A., Rahimy, E., Blumer, K., Coz, D., Hammel, N., Krause, J., Narayanaswamy, A., Rastegar, Z., Wu, D., et al. (2019) · Ophthalmology · doi:10.1016/j.ophtha.2018.11.016 · Cited by 418
Distilling a neural network into a soft decision tree
Frosst, N., & Hinton, G. (2017) · arXiv · doi:10.48550/arxiv.1711.09784 · Cited by 407 · cited 1×
“…from building inherently interpretable models (Kim et al., 2014) to building a proxy model for explaining a more complex model (Ribeiro et al., 2016; Frosst and Hinton, 2017) to explaining inner mechanics of mostly uninterpretable neural networks (Sun-dararajan et al., 2017; Bach et al., 2015).”
BackgroundReducing Gender Bias in Abusive Language Detection
Park, J. H., Shin, J., & Fung, P. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1302 · Cited by 323 · cited 8×
“Unlike previous approaches (Park et al., 2018; Dixon et al., 2018; Madras et al., 2018), our method does not degrade classifier performance (it even improves) in terms of all reported metrics.”
BackgroundMethodInfluentialDid the Model Understand the Question?
Mudrakarta, P. K., Taly, A., Sundararajan, M., & Dhamdhere, K. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1176 · Cited by 166 · cited 3×
“Hence, interpretability methods tailored for text are quite sparse (Mudrakarta et al., 2018; Jia and Liang, 2017; Murdoch et al., 2018).”
BackgroundMethod
Cited by
Erion, G., Janizek, J. D., Sturmfels, P., Lundberg, S. M., & Lee, S.-I. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00343-w · Cited by 300
“While feature attributions have previously been used in training (more details in ‘Previous attribution priors’ in Methods) 9,12 , our approach offers two novel components.”
BackgroundMethodSchramowski, P., Stammer, W., Teso, S., Brugger, A., Herbert, F., Shao, X., Luigs, H.-G., Mahlein, A.-K., & Kersting, K. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0212-3 · Cited by 207
In-Processing Modeling Techniques for Machine Learning Fairness: A Survey
Wan, M., Zha, D., Liu, N., & Zou, N. (2023) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3551390 · Cited by 99
“Also, local interpretation could serve as regularization terms to achieve fairness by enforcing the predictions to be less dependent on the sensitive attributes [57, 72].”
BackgroundShortcut Learning of Large Language Models in Natural Language Understanding
Du, M., He, F., Zou, N., Tao, D., & Hu, X. (2024) · Communications of the ACM · doi:10.1145/3596490 · Cited by 94
“attribution explanations with rationale annotations created by domain experts, to enforce the model to make the right predictions for the right reasons (Liu and Avci, 2019).”
BackgroundAugmenting interpretable models with large language models during training
Singh, C., Askari, A., Caruana, R., & Gao, J. (2023) · Nature Communications · doi:10.1038/s41467-023-43713-1 · Cited by 63
“Slightly more related are works which aim to explain feature interactions or transformations in neural networks [69–71], but these works fail to explain the model as a whole and are again less reliable than having a fully transparent model.”
Metadata
- Authors
- Liu, F., & Avci, B.
- Venue
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Published
- 2019
- Pages
- 6274-6283
- OpenAlex
- W2951576127
- Cited by
- 91
- References
- 29
- FWCI
- 5.01
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.