The DOI matches its DataCite record; 30 of the 31 references it cites were found in a registry or an index.
Grammar as a Foreign Language
Open accessAbstract
Syntactic constituency parsing is a fundamental problem in natural language processing and has been the subject of intensive research and engineering for decades. As a result, the most accurate parsers are domain specific, complex, and inefficient. In this paper we show that the domain agnostic attention-enhanced sequence-to-sequence model achieves state-of-the-art results on the most widely used syntactic constituency parsing dataset, when trained on a large synthetic corpus that was annotated using existing parsers. It also matches the performance of standard parsers when trained only on a small human-annotated dataset, which shows that this model is highly data-efficient, in contrast to sequence-to-sequence models without the attention mechanism. Our parser is also fast, processing over a hundred sentences per second with an unoptimized CPU implementation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, RNN sequence-to-sequence models have not been able to attain state-of-the-art results in small-data regimes [37].”
From Attention Is All You Need (2017) · cites this paper as [37]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Self-citationBuilding a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Accurate unlexicalized parsing
Klein, D., & Manning, C. D. (2003) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1075096.1075150 · Cited by 3,055
Generating Sequences With Recurrent Neural Networks
Graves, A. (2013) · arXiv · doi:10.48550/arxiv.1308.0850 · Cited by 2,894
Recurrent Continuous Translation Models
Kalchbrenner, N., & Blunsom, P. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1176 · Cited by 1,277
(2011) · Cited by 1,192
Hovy, E., Marcus, M., Palmer, M., Ramshaw, L., & Weischedel, R. (2006) · Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers on XX - NAACL '06 · doi:10.3115/1614049.1614064 · Cited by 806
Learning Accurate, Compact, and Interpretable Tree Annotation
Petrov, S., Barrett, L., Thibaux, R., & Klein, D. (2006) · Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the ACL - ACL '06 · doi:10.3115/1220175.1220230 · Cited by 793
Self-citationEffective self-training for parsing
McClosky, D., Eugene Charniak, D., & Johnson, M. (2006) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/1220835.1220855 · Cited by 603
Learning to ExecutePreprint
Zaremba, W., & Sutskever, I. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.4615 · Cited by 525
Self-citationEnd-to-end Continuous Speech Recognition using Attention-based Recurrent NN: First ResultsPreprint
Chorowski, J., Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.1602 · Cited by 475
Algorithms for Deterministic Incremental Dependency Parsing
Nivre, J. (2008) · Computational Linguistics · doi:10.1162/coli.07-056-r1-07-027 · Cited by 440
Incremental parsing with the perceptron algorithm
Collins, M., & Brian Roark, A. (2004) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1218955.1218970 · Cited by 373
A Linear Observed Time Statistical Parser Based on Maximum Entropy ModelsPreprint
Ratnaparkhi, A. (1997) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9706014 · Cited by 210
Fast and Accurate Shift-Reduce Constituent Parsing
Zhu, M., Zhang, Y., Chen, W., Zhang, M., & Zhu, J. (2013) · doi:10.13140/2.1.2791.2961 · Cited by 166
Overview of the 2012 Shared Task on Parsing the Web
Petrov, S., & McDonald, R. (2012) · Cited by 158
Self-citationDiscriminative training of a neural network statistical parser
Henderson, J. (2004) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1218955.1218968 · Cited by 106
Addressing the Rare Word Problem in Neural Machine TranslationPreprint
Luong, M.-T., Sutskever, I., Le, Q. V., Vinyals, O., & Zaremba, W. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1410.8206 · Cited by 101
Self-citationSelf-training PCFG grammars with latent annotations across languages
Huang, Z., & Harper, M. P. (2009) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1699571.1699621 · Cited by 89
Products of Random Latent Variable Grammars
Petrov, S. (2010) · Cited by 87
Self-citationInducing history representations for broad coverage statistical parsing
Henderson, J. (2003) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/1073445.1073459 · Cited by 86
Judge, J., Cahill, A., & van Genabith, J. (2006) · International Conference on Computational Linguistics · doi:10.3115/1220175.1220238 · Cited by 81
Three Generative, Lexicalised Models for Statistical ParsingPreprint
Collins, M. (1997) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9706022 · Cited by 44
Constituent Parsing with Incremental Sigmoid Belief Networks
Titov, I., & Henderson, J. (2007) · Cited by 43
Self-training with Products of Latent Variable Grammars
Huang, Z., Harper, M., & Petrov, S. (2010) · Cited by 40
Self-citationAmbiguity-aware Ensemble Training for Semi-supervised Dependency Parsing
Li, Z., Zhang, M., & Chen, W. (2014) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/v1/p14-1043 · Cited by 33
Sparser, Better, Faster GPU Parsing
Hall, D., Berg-Kirkpatrick, T., & Klein, D. (2014) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/v1/p14-1020 · Cited by 18
Incremental Sigmoid Belief Networks for Grammar Learning
Henderson, J., & Titov, I. (2010) · Cited by 13
Research works printed but not linked: 5
- [2][2] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 , 2014. - · arXiv
- [4][4] S´ ebastien Jean, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. On using very large target vocabulary for neural machine translation. arXiv preprint arXiv:1412.2007 , 2014. - · arXiv
- [24][24] Ronan Collobert. Deep learning for efficient discriminative parsing. In International Conference on Artificial Intelligence and Statistics , 2011. -
- [31][31] Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. Show and tell: A neural image caption generator. arXiv preprint arXiv:1411.4555 , 2014. - · arXiv
- [32][32] Zoubin Ghahramani. A neural network for learning how to parse tree adjoining grammar. B.S.Eng Thesis, University of Pennsylvania, 1990.
Cited by
Sorted by citations
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Self-citationDeep Learning with Differential Privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978318 · Cited by 6,578
Hierarchical Attention Networks for Document Classification
Yang, Z., Yang, D., Dyer, C., He, X., Smola, A. J., & Hovy, E. H. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1174 · Cited by 5,022
Recent Trends in Deep Learning Based Natural Language Processing [Review Article]
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2840738 · Cited by 2,921
Anchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162
“In Table 2, we compute anchors for a multi-layer RNN encoder/attention-based decoder translation system (Vinyals et al. 2015) trained on English-Portuguese parallel corpora.”
Method
Metadata
- Authors
- Vinyals, O., Kaiser, L., Koo, T., Petrov, S., Sutskever, I., & Hinton, G.
- Affiliations
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2014
- Volume
- 28
- Pages
- 2773-2781
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1412.7449
- OpenAlex
- W1869752048
- Cited by
- 464
- References
- 31
- Citations by year
- 2025: 2 · 2024: 11 · 2023: 26 · 2022: 24 · 2021: 42 · 2020: 42 · 2019: 71 · 2018: 88 · 2017: 72 · 2016: 57 · 2015: 29
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- neural information processing systems
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
- Keywords
- synthetic training data, data efficiency, natural language parsing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
