The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Explaining Recurrent Neural Network Predictions in Sentiment Analysis
Open accessAbstract
Recently, a technique called Layer-wise Relevance Propagation (LRP) was shown to deliver insightful explanations in the form of input space relevances for understanding feed-forward neural network classification decisions.In the present work, we extend the usage of LRP to recurrent neural networks.We propose a specific propagation rule applicable to multiplicative connections as they arise in recurrent network architectures such as LSTMs and GRUs.We apply our technique to a word-based bi-directional LSTM model on a five-class sentiment prediction task, and evaluate the resulting LRP relevances both qualitatively and quantitatively, obtaining better results than a gradient-based related method which was used in previous work.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Bidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172
Learning to Forget: Continual Prediction with LSTM
Gers, F. A., Schmidhuber, J., & Cummins, F. (2000) · Neural Computation · doi:10.1162/089976600300015015 · Cited by 6,122
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524
Self-citationGevrey, M., Dimopoulos, I. F., & Lek, S. (2003) · Ecological Modelling · doi:10.1016/s0304-3800(02)00257-0 · Cited by 1,263
Evaluating the Visualization of What a Deep Neural Network Has Learned
Samek, W., Binder, A., Montavon, G., Lapuschkin, S., & Muller, K.-R. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2599820 · Cited by 1,164
Self-citationUse of some sensitivity criteria for choosing networks with good generalization ability
Dimopoulos, Y., Bourret, P., & Lek, S. (1995) · Neural Processing Letters · doi:10.1007/bf02309007 · Cited by 267
Research works printed but not linked: 16
- Leila Arras, Franziska Horn, Grégoire Montavon, Klaus-Robert Müller, and Wojciech Samek. 2016a. Explaining Predictions of Non-Linear Classifiers in NLP. In Proceedings of the ACL 2016 Workshop on Representation Learning for NLP. Association for Computational Linguistics, pages 1-7.
- Leila Arras, Franziska Horn, Grégoire Montavon, Klaus-Robert Müller, and Wojciech Samek. 2016b. "What is Relevant in a Text Document?": An In- terpretable Machine Learning Approach. arXiv 1612.07843. · arXiv
- Kyunghyun Cho, Bart van Merrienboer, Caglar Gul- cehre, Dzmitry Bahdanau, Fethi Bougares, Hol- ger Schwenk, and Yoshua Bengio. 2014. Learn- ing Phrase Representations using RNN Encoder- Decoder for Statistical Machine Translation. In Pro- ceedings of the 2014 Conference on Empirical Meth- ods in Natural Language Processing (EMNLP). Association for Computational Linguistics, pages 1724-1734.
- Misha Denil, Alban Demiraj, and Nando de Freitas. 2014. Extraction of Salient Sentences from Labelled Documents. arXiv 1412.6815. · arXiv
- Will Landecker, Michael D. Thomure, Luís M. A. Bet- tencourt, Melanie Mitchell, Garrett T. Kenyon, and Steven P. Brumby. 2013. Interpreting Individual Classifications of Hierarchical Networks. In IEEE Symposium on Computational Intelligence and Data Mining (CIDM). pages 32-38.
- Sebastian Lapuschkin, Alexander Binder, Grégoire Montavon, Klaus-Robert Müller, and Wojciech Samek. 2016. The Layer-wise Relevance Propaga- tion Toolbox for Artificial Neural Networks. Jour- nal of Machine Learning Research 17(114):1-5.
- Jiwei Li, Xinlei Chen, Eduard Hovy, and Dan Juraf- sky. 2016a. Visualizing and Understanding Neural Models in NLP. In Proceedings of the 2016 Con- ference of the North American Chapter of the Asso- ciation for Computational Linguistics: Human Lan- guage Technologies (NAACL-HLT). pages 681-691.
- Jiwei Li, Will Monroe, and Dan Jurafsky. 2016b. Un- derstanding Neural Networks through Representa- tion Erasure. arXiv 1612.08220. · arXiv
- Saif M. Mohammad. 2017. Challenges in Sentiment Analysis. In Erik Cambria, Dipankar Das, Sivaji Bandyopadhyay, and Antonio Feraco, editors, A Practical Guide to Sentiment Analysis. Springer In- ternational Publishing, pages 61-83.
- Grégoire Montavon, Wojciech Samek, and Klaus- Robert Müller. 2017b. Methods for Interpreting and Understanding Deep Neural Networks. arXiv 1706.07979. · arXiv
- James Murdoch and Arthur Szlam. 2017. Automatic Rule Extraction from Long Short Term Memory Networks. In International Conference on Learning Representations Conference Track (ICLR).
- Preslav Nakov, Alan Ritter, Sara Rosenthal, Fabrizio Sebastiani, and Veselin Stoyanov. 2016. SemEval- 2016 Task 4: Sentiment Analysis in Twitter. In Pro- ceedings of the 10th International Workshop on Se- mantic Evaluation (SemEval-2016). Association for Computational Linguistics, pages 1-18.
- Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Ex- plaining the Predictions of Any Classifier. In Pro- ceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Min- ing. pages 1135-1144.
- Mike Schuster and Kuldip K. Paliwal. 1997. Bidirec- tional Recurrent Neural Networks. IEEE Transac- tions on Signal Processing 45(11):2673-2681.
- Richard Socher, Alex Perelygin, Jean Y. Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. 2013. Recursive Deep Mod- els for Semantic Compositionality Over a Sentiment Treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Pro- cessing (EMNLP). Association for Computational Linguistics, pages 1631-1642.
- Matthew D. Zeiler and Rob Fergus. 2014. Visualiz- ing and Understanding Convolutional Networks. In Computer Vision ECCV 2014: 13th European Con- ference. pages 818-833.
Cited by
Sorted by citations
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Methods for interpreting and understanding deep neural networks
Montavon, G., Samek, W., & Müller, K.-R. (2018) · Digital Signal Processing · doi:10.1016/j.dsp.2017.10.011 · Cited by 2,665
Self-citationA Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI
Tjoa, E., & Guan, C. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3027314 · Cited by 2,524
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3060483 · Cited by 1,477
“Other neural network architectures have a more conventional structure but still require a nontrivial adaptation of existing explanation methods, for example, extensions of LRP have been proposed to deal with the special LSTM blocks in recurrent neural networks [9], [11] or to handle attention units in the context of n…”
BackgroundMethodSelf-citationSamek, W., Wiegand, T., & Müller, K.-R. (2017) · arXiv · doi:10.48550/arxiv.1708.08296 · Cited by 1,080
Self-citation
Metadata
- Authors
- Arras, L., Montavon, G., Müller, K.-R., & Samek, W.
- Affiliations
- Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute (DE)
- Technische Universität Berlin (DE)
- Freie Universität Berlin (DE)
- Venue
- Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis
- Published
- 2017
- Pages
- 159-168
- OpenAlex
- W2639206954
- Cited by
- 361
- References
- 5
- Citations by year
- 2026: 6 · 2025: 22 · 2024: 42 · 2023: 49 · 2022: 47 · 2021: 63 · 2020: 58 · 2019: 42 · 2018: 26 · 2017: 6
- FWCI
- 16.46
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Deutsche Forschungsgemeinschaft
- Bundesministerium für Bildung und Forschung
- Institute for Information and Communications Technology Promotion (2017-0-00451)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- layer-wise relevance propagation, explainable AI, recurrent neural networks, sentiment analysis, LSTM, GRU, bidirectional LSTM
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
