The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Pointing the Unknown Words
Open accessAbstract
The problem of rare and unknown words is an important issue that can potentially effect the performance of many NLP systems, including traditional count-based and deep learning models.We propose a novel way to deal with the rare and unseen words for the neural network models using attention.Our model uses two softmax layers in order to predict the next word in conditional language models: one predicts the location of a word in the source sentence, and the other predicts a word in the shortlist vocabulary.At each timestep, the decision of which softmax layer to use is adaptively made by an MLP which is conditioned on the context.We motivate this work from a psychological evidence that humans naturally have a tendency to point towards objects in the context or the environment when the name of an object is not known.Using our proposed model, we observe improvements on two tasks, neural machine translation on the Europarl English to French parallel corpora and text summarization on the Gigaword dataset.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 2×
“In the decoder, we usually use gated recurrent unit (GRU) (Cho et al., 2014; Chung et al., 2014).”
MethodSelf-citationNeural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Self-citationEmpirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Self-citationBidirectional recurrent neural networks
Schuster, M., & Paliwal, K. K. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650093 · Cited by 10,172 · cited 6×
“The forward RNN reads the input sequence x = ( x 1 , . . . , x T ) in the left-to-right direction, resulting in a sequence of hidden states ( −→ h 1 , . . . , −→ h T ) .”
MethodInfluentialADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322
On the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310
Self-citationDeep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196
Generating Sequences With Recurrent Neural Networks
Graves, A. (2013) · arXiv · doi:10.48550/arxiv.1308.0850 · Cited by 2,894
A Neural Attention Model for Abstractive Sentence Summarization
Rush, A. M., Chopra, S., & Weston, J. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1044 · Cited by 2,656
Cited by
Get To The Point: Summarization with Pointer-Generator Networks
See, A., Liu, P. J., & Manning, C. D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1099 · Cited by 3,775
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
Nallapati, R., Zhou, B., dos Santos, C., Gulcehre, C., & Xiang, B. (2016) · Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1028 · Cited by 2,361
Self-citationIncorporating Copying Mechanism in Sequence-to-Sequence Learning
Gu, J., Lu, Z., Li, H., & Li, V. O. K. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1154 · Cited by 1,446
Neural Summarization by Extracting Sentences and Words
Cheng, J., & Lapata, M. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1046 · Cited by 790
Towards VQA Models That Can Read
Singh, A., Natarajan, V., Shah, M., Jiang, Y., Chen, X., Batra, D., Parikh, D., & Rohrbach, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00851 · Cited by 711
Challenges in Data-to-Document Generation
Wiseman, S., Shieber, S., & Rush, A. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1239 · Cited by 524
The Natural Language Decathlon: Multitask Learning as Question Answering
McCann, B., Keskar, N. S., Xiong, C., & Socher, R. (2018) · arXiv · doi:10.48550/arxiv.1806.08730 · Cited by 448
Data Recombination for Neural Semantic Parsing
Jia, R., & Liang, P. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1002 · Cited by 441
Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems
Wu, C.-S., Madotto, A., Hosseini-Asl, E., Xiong, C., Socher, R., & Fung, P. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1078 · Cited by 415
Key-Value Retrieval Networks for Task-Oriented Dialogue
Eric, M., Krishnan, L., Charette, F., & Manning, C. D. (2017) · Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue · doi:10.18653/v1/w17-5506 · Cited by 388
Coarse-to-Fine Decoding for Neural Semantic Parsing
Dong, L., & Lapata, M. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1068 · Cited by 374
Neural Question Generation from Text: A Preliminary Study
Zhou, Q., Yang, N., Wei, F., Tan, C., Bao, H., & Zhou, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-73618-1_56 · Cited by 346
Scene Text Visual Question Answering
Biten, A. F., Tito, R., Mafla, A., Gomez, L., Rusinol, M., Jawahar, C. V., Valveny, E., & Karatzas, D. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00439 · Cited by 293
Selective Encoding for Abstractive Sentence Summarization
Zhou, Q., Yang, N., Wei, F., & Zhou, M. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1101 · Cited by 265
A Survey of Knowledge-enhanced Text Generation
Yu, W., Zhu, C., Li, Z., Hu, Z., Wang, Q., Ji, H., & Jiang, M. (2022) · ACM Computing Surveys · doi:10.1145/3512467 · Cited by 257
A Graph-to-Sequence Model for AMR-to-Text Generation
Song, L., Zhang, Y., Wang, Z., & Gildea, D. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1150 · Cited by 216
“Copy We find that most of the open-class tokens in a graph also appear in the corresponding sentence, and thus adopt the copy mechanism (Gulcehre et al., 2016; Gu et al., 2016) to solve”
BackgroundMethodInfluentialControlling Output Length in Neural Encoder-Decoders
Kikuchi, Y., Neubig, G., Sasano, R., Takamura, H., & Okumura, M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1140 · Cited by 222
ALFWorld: Aligning Text and Embodied Environments for Interactive Learning
Shridhar, M., Yuan, X., Côté, M.-A., Bisk, Y., Trischler, A., & Hausknecht, M. (2020) · arXiv · doi:10.48550/arxiv.2010.03768 · Cited by 80
“Our decoder consists of an embedding layer, a transformer block and a pointer softmax mechanism (Gulcehre et al., 2016).”
MethodInfluentialSelf-Attention Architectures for Answer-Agnostic Neural Question Generation
Scialom, T., Piwowarski, B., & Staiano, J. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1604 · Cited by 70
“As the questions generated from a given context usually tend to refer to specific phrasing or entities appearing therein, Gulcehre et al. (2016) pro-pose using a pointing mechanism (called pointer-softmax ) to select words to be copied from the source sentence; intuitively, such method is of…”
Conversational Query Understanding Using Sequence to Sequence Modeling
Ren, G., Ni, X., Malik, M., & Ke, Q. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186083 · Cited by 55
Data-to-text generation with macro planning
Puduppully, R., & Lapata, M. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00381 · Cited by 53
Generating Better Search Engine Text Advertisements with Deep Reinforcement Learning
Hughes, J. W., Chang, K.-H., & Zhang, R. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330754 · Cited by 43
An End-to-end Approach for Handling Unknown Slot Values in Dialogue State Tracking
Xu, P., & Hu, Q. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1134 · Cited by 38
Revisiting Challenges in Data-to-Text Generation with Fact Grounding
Wang, H. (2019) · Proceedings of the 12th International Conference on Natural Language Generation · doi:10.18653/v1/w19-8639 · Cited by 34
Mind The Facts: Knowledge-Boosted Coherent Abstractive Text Summarization
Gunel, B., Zhu, C., Zeng, M., & Huang, X. (2020) · arXiv · doi:10.48550/arxiv.2006.15435 · Cited by 28
Question Generation With Doubly Adversarial Nets
Bao, J., Gong, Y., Duan, N., Zhou, M., & Zhao, T. (2018) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2018.2859777 · Cited by 14
History-Aware Hierarchical Transformer for Multi-session Open-domain Dialogue System
Zhang, T., Liu, Y., Li, B., Zeng, Z., Wang, P., You, Y., Miao, C., & Cui, L. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.247 · Cited by 14
Metadata
- Authors
- Gulcehre, C., Ahn, S., Nallapati, R., Zhou, B., & Bengio, Y.
- Venue
- Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2016
- Pages
- 140-149
- OpenAlex
- W2962944953
- Cited by
- 478
- References
- 28
- FWCI
- 45.02
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.