The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
Harnessing Deep Neural Networks with Logic Rules
Open accessAbstract
Combining deep neural networks with structured logic rules is desirable to harness flexibility and reduce uninterpretability of the neural models.We propose a general framework capable of enhancing various types of neural networks (e.g., CNNs and RNNs) with declarative first-order logic rules.Specifically, we develop an iterative distillation method that transfers the structured information of logic rules into the weights of neural networks.We deploy the framework on a CNN for sentiment analysis, and an RNN for named entity recognition.With a few highly intuitive rules, we obtain substantial improvements and achieve state-of-the-art or comparable results to previous best-performing systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
Mining and summarizing customer reviews
Hu, M., & Liu, B. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014073 · Cited by 7,793
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
ADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322
Natural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962
Distributed Representations of Sentences and Documents
Le, Q. V., & Mikolov, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.4053 · Cited by 4,934
Neural Architectures for Named Entity Recognition
Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., & Dyer, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1030 · Cited by 4,401
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258
Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images
Nguyen, A., Yosinski, J., & Clune, J. (2014) · arXiv · doi:10.48550/arxiv.1412.1897 · Cited by 3,258
Human-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817
Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition
Tjong Kim Sang, E. F., & De Meulder, F. (2003) · Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003 - · doi:10.3115/1119176.1119195 · Cited by 2,788
Richardson, M., & Domingos, P. (2006) · Machine Learning · doi:10.1007/s10994-006-5833-1 · Cited by 2,676
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
Ma, X., & Hovy, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1101 · Cited by 2,567 · cited 4×
“Further integration of the list rule (Row 3) provides a second boost in performance, achieving an F1 score very close to the best-performing systems including Joint-NER-EL (Luo et al., 2015) (Row 8), a probabilistic graphical model optimizing NER and entity linking jointly with massive external resources, and BLSTM-CR…”
MethodInfluentialSelf-citationStochastic Backpropagation and Approximate Inference in Deep Generative Models
Rezende, D. J., Mohamed, S., & Wierstra, D. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1401.4082 · Cited by 2,405
Pang, B., & Lee, L. (2005) · Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics - ACL '05 · doi:10.3115/1219840.1219855 · Cited by 2,129
Buciluǎ, C., Caruana, R., & Niculescu-Mizil, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150464 · Cited by 2,068
Named Entity Recognition with Bidirectional LSTM-CNNs
Chiu, J. P. C., & Nichols, E. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00104 · Cited by 1,909
Understanding deep image representations by inverting them
Mahendran, A., & Vedaldi, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299155 · Cited by 1,827
Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition
Sang, E. T. K. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0209010 · Cited by 1,560
Pang, B., & Lee, L. (2005) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0506075 · Cited by 678
Deep Convolutional Inverse Graphics Network
Kulkarni, T. D., Whitney, W., Kohli, P., & Tenenbaum, J. B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.03167 · Cited by 653
Posterior Regularization for Structured Latent Variable Models
Ganchev, K., Graça, J., Gillenwater, J., & Taskar, B. (2010) · Cited by 440
Refinement of approximate domain theories by knowledge-based neural networks
Towell, G. G., SHAVLIK, J. W., & Noordewier, M. O. (1990) · Cited by 369
(2013) · Cited by 326
Unifying distillation and privileged information
Lopez-Paz, D., Bottou, L., Schölkopf, B., & Vapnik, V. (2015) · arXiv · doi:10.48550/arxiv.1511.03643 · Cited by 270 · cited 2×
“, the privileged information setting (Lopez-Paz et al., 2016)) while still enjoying the benefit of integration.”
BackgroundMethodHinge-Loss Markov Random Fields and Probabilistic Soft LogicPreprint
Bach, S. H., Broecheler, M., Huang, B., & Getoor, L. C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.04406 · Cited by 270
Posterior Regularization for Structured Latent Variable Models
GanchevKuzman, GraçaJoão, GillenwaterJennifer, & TaskarBen (2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1859918 · Cited by 201
Multichannel Variable-Size Convolution for Sentence Classification
Yin, W., & Schütze, H. (2015) · Proceedings of the Nineteenth Conference on Computational Natural Language Learning · doi:10.18653/v1/k15-1021 · Cited by 162
Context-aware Learning for Sentence-level Sentiment Analysis with Posterior Regularization
Yang, B., & Cardie, C. (2014) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/v1/p14-1031 · Cited by 161
MGNC-CNN: A Simple Approach to Exploiting Multiple Word Embeddings for Sentence Classification
Zhang, Y., Roller, S., & Wallace, B. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1178 · Cited by 130
Fast relational learning using bottom clause propositionalization with artificial neural networks
França, M. V. M., Zaverucha, G., & Garcez, A. D. (2013) · Machine Learning · doi:10.1007/s10994-013-5392-1 · Cited by 125
Bayesian Inference with Posterior Regularization and Applications to Infinite Latent SVMs
Zhu, J., Chen, N., & Xing, E. P. (2018) · Figshare · doi:10.5555/2627435.2638592 · Cited by 114
Self-citationLearning from measurements in exponential families
Liang, P., Jordan, M. I., & Klein, D. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553457 · Cited by 111
Bayesian representation learning with oracle constraintsPreprint
Karaletsos, T., Belongie, S., & Rätsch, G. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.05011 · Cited by 83
Structure compilation: trading structure for features
Liang, P., Daumé, H., & Klein, D. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390231 · Cited by 75
Joint Named Entity Recognition and Disambiguation
Luo, G., Huang, X., Lin, C.-Y., & Nie, Z. (2015) · Cited by 69
Latent Topic Networks: A Versatile Probabilistic Programming Framework for Topic Models
Foulds, J. R., Kumar, S. H., & Getoor, L. C. (2015) · International Conference on Machine Learning · Cited by 23
Structured VAEs: Composing Probabilistic Graphical Models and Variational AutoencodersPreprint
Johnson, M., Duvenaud, D., Wiltschko, A. B., Datta, S. R., & Adams, R. P. (2016) · arXiv (Cornell University) · Cited by 14
Cited by
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
Ma, X., & Hovy, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1101 · Cited by 2,567
“Huang et al. (2015) used BLSTM for word-level representations and CRF for jointly label decoding, which is similar to our model.”
MethodResultInfluentialSelf-citationVisual interpretability for deep learning: a survey
Zhang, Q.-S., & Zhu, S.-C. (2018) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1700808 · Cited by 950
Interpretable Convolutional Neural Networks
Zhang, Q., Wu, Y. N., & Zhu, S.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00920 · Cited by 833
von Rueden, L., Mayer, S., Beckh, K., Georgiev, B., Giesselbach, S., Heese, R., Kirsch, B., Walczak, M., Pfrommer, J., Pick, A., et al. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3079836 · Cited by 793
On Interpretability of Artificial Neural Networks: A Survey
Fan, F.-L., Xiong, J., Li, M., & Wang, G. (2021) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2021.3066428 · Cited by 561
Metadata
- Authors
- Hu, Z., Ma, X., Liu, Z., Hovy, E., & Xing, E.
- Venue
- Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2016
- Pages
- 2410-2420
- OpenAlex
- W2963687836
- Cited by
- 545
- References
- 48
- FWCI
- 34.67
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.