Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 257

The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.

"What is relevant in a text document?": An interpretable machine learning approach

Open access
Arras, Leila, Horn, Franziska, Montavon, Grégoire, Müller, Klaus-Robert, Samek, Wojciech·2017·PLOS ONE
DOI 10.1371/journal.pone.0181142
Open this paper in CoChat — free account

Abstract

Text documents can be described by a number of abstract concepts such as semantic category, writing style, or sentiment. Machine learning (ML) models have been trained to automatically map documents to these abstract concepts, allowing to annotate very large text collections, more than could be processed by a human in a lifetime. Besides predicting the text's category very accurately, it is also highly desirable to understand how and why the categorization process takes place. In this paper, we demonstrate that such understanding can be achieved by tracing the classification decision back to individual words using layer-wise relevance propagation (LRP), a recently developed technique for explaining predictions of complex non-linear classifiers. We train two word-based ML models, a convolutional neural network (CNN) and a bag-of-words SVM classifier, on a topic categorization task and adapt the LRP method to decompose the predictions of these models onto words. Resulting scores indicate how much individual words contribute to the overall classification decision. This enables one to distill relevant information from text documents without an explicit semantic information extraction step. We further use the word-wise relevance scores for generating novel vector-based document representations which capture semantic information. Based on these document vectors, we introduce a measure of model explanatory power and show that, although the SVM and CNN models perform similarly in terms of classification accuracy, the latter exhibits a higher level of explainability which makes it more comprehensible for humans and potentially more useful for other applications.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0181142

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 13 references

Research works printed but not linked: 26

  1. KS Jones, 1972, A statistical interpretation of term specificity and its application in retrieval, Journal of Documentation, 28, 11, 10.1108/eb026526 · DOI · Paper page
  2. Hasan KS, Ng V. Conundrums in Unsupervised Keyphrase Extraction: Making Sense of the State-of-the-Art. In: Proceedings of the 23rd International Conference on Computational Linguistics: Posters (COLING); 2010. p. 365–373.
  3. Y Bengio, 2003, A Neural Probabilistic Language Model, Journal of Machine Learning Research (JMLR), 3, 1137
  4. R Collobert, 2011, Natural Language Processing (Almost) from Scratch, Journal of Machine Learning Research (JMLR), 12, 2493
  5. Socher R, Perelygin A, Wu J, Chuang J, Manning CD, Ng A, et al. Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics; 2013. p. 1631–1642. · DOI · Paper page
  6. Zeiler MD, Fergus R. Visualizing and Understanding Convolutional Networks. In: Computer Vision—ECCV 2014: 13th European Conference; 2014. p. 818–833. · DOI · Paper page
  7. Landecker W, Thomure MD, Bettencourt LMA, Mitchell M, Kenyon GT, Brumby SP. Interpreting Individual Classifications of Hierarchical Networks. In: IEEE Symposium on Computational Intelligence and Data Mining (CIDM); 2013. p. 32–38. · DOI
  8. Arbabzadah F, Montavon G, Müller KR, Samek W. Identifying Individual Facial Expressions by Deconstructing a Neural Network. In: Pattern Recognition—38th German Conference, GCPR 2016. vol. 9796 of LNCS. Springer; 2016. p. 344–354. · DOI
  9. I Sturm, 2016, Interpretable Deep Neural Networks for Single-Trial EEG Classification, Journal of Neuroscience Methods, 274, 141, 10.1016/j.jneumeth.2016.10.008 · DOI · Paper page
  10. Lapuschkin S, Binder A, Montavon G, Müller KR, Samek W. Analyzing Classifiers: Fisher Vectors and Deep Neural Networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016. p. 2912–2920. · DOI
  11. Poulin B, Eisner R, Szafron D, Lu P, Greiner R, Wishart DS, et al. Visual Explanation of Evidence in Additive Classifiers. In: Proceedings of the 18th Conference on Innovative Applications of Artificial Intelligence (IAAI). AAAI Press; 2006. p. 1822–1829.
  12. D Baehrens, 2010, How to Explain Individual Classification Decisions, Journal of Machine Learning Research (JMLR), 11, 1803
  13. E Strumbelj, 2010, An Efficient Explanation of Individual Classifications using Game Theory, Journal of Machine Learning Research (JMLR), 11, 1
  14. Turner R. A Model Explanation System: Latest Updates and Extensions. arXiv. 2016;1606.09517.
  15. M Gevrey, 2003, Review and comparison of methods to study the contribution of variables in artificial neural network models, Ecological Modelling, 160, 249, 10.1016/S0304-3800(02)00257-0 · DOI · Paper page
  16. Montavon G, Samek W, Müller KR. Methods for Interpreting and Understanding Deep Neural Networks. arXiv. 2017;1706.07979.
  17. Denil M, Demiraj A, de Freitas N. Extraction of Salient Sentences from Labelled Documents. arXiv. 2014;1412.6815.
  18. Li J, Chen X, Hovy E, Jurafsky D. Visualizing and Understanding Neural Models in NLP. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT); 2016. p. 681–691. · DOI
  19. Arras L, Horn F, Montavon G, Müller KR, Samek W. Explaining Predictions of Non-Linear Classifiers in NLP. In: Proceedings of the 1st Workshop on Representation Learning for NLP. Association for Computational Linguistics; 2016. p. 1–7. · DOI
  20. Ding Y, Liu Y, Luan H, Sun, M. Visualizing and Understanding Neural Machine Translation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics; 2017. · DOI
  21. Kim Y. Convolutional Neural Networks for Sentence Classification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP); 2014. p. 1746–1751. · DOI · Paper page
  22. Mnih A, Hinton G. Three New Graphical Models for Statistical Language Modelling. In: Proceedings of the International Conference on Machine Learning (ICML); 2007. p. 641–648. · DOI · Paper page
  23. Mnih A, Teh YW. A fast and simple algorithm for training neural probabilistic language models. In: Proceedings of the International Conference on Machine Learning (ICML); 2012. p. 1751–1758.
  24. S Lapuschkin, 2016, The Layer-wise Relevance Propagation Toolbox for Artificial Neural Networks, Journal of Machine Learning Research, 17, 1
  25. Zhang X, Zhao J, LeCun Y. Character-level Convolutional Networks for Text Classification. In: Advances in Neural Information Processing Systems 28 (NIPS); 2015. p. 649–657.
  26. C Nadeau, 2003, Inference for the Generalization Error, Machine Learning, 52, 239, 10.1023/A:1024068626366 · DOI · Paper page

Cited by

Metadata

Authors
Arras, L., Horn, F., Montavon, G., Müller, K.-R., & Samek, W.
Venue
PLOS ONE
Publisher
Public Library of Science (PLoS)
Published
2017-08-11
Volume
12
Issue
8
Pages
e0181142
ISSN
1932-6203 (ISSN-L)
DOI
10.1371/journal.pone.0181142
OpenAlex
W2561412020
Cited by
257
References
13
FWCI
19.35
Type
journal article
Language
en
Open access
gold
Licence
creativecommons.org/licenses/by/4.0/ (vor)
Funders
  • German Ministry for Education and Research (Berlin Big Data Center BBDC (01IS14013A))
  • National Research Foundation of Korea (BK21 program)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.