The DOI matches its DataCite record; 30 of the 31 references it cites were found in a registry or an index.
Grammar as a Foreign Language
Open accessAbstract
Syntactic constituency parsing is a fundamental problem in natural language processing and has been the subject of intensive research and engineering for decades. As a result, the most accurate parsers are domain specific, complex, and inefficient. In this paper we show that the domain agnostic attention-enhanced sequence-to-sequence model achieves state-of-the-art results on the most widely used syntactic constituency parsing dataset, when trained on a large synthetic corpus that was annotated using existing parsers. It also matches the performance of standard parsers when trained only on a small human-annotated dataset, which shows that this model is highly data-efficient, in contrast to sequence-to-sequence models without the attention mechanism. Our parser is also fast, processing over a hundred sentences per second with an unoptimized CPU implementation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, RNN sequence-to-sequence models have not been able to attain state-of-the-art results in small-data regimes [37].”
From Attention Is All You Need (2017) · cites this paper as [37]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Self-citationBuilding a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Accurate unlexicalized parsing
Klein, D., & Manning, C. D. (2003) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1075096.1075150 · Cited by 3,055
Generating Sequences With Recurrent Neural Networks
Graves, A. (2013) · arXiv · doi:10.48550/arxiv.1308.0850 · Cited by 2,894
Recurrent Continuous Translation Models
Kalchbrenner, N., & Blunsom, P. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1176 · Cited by 1,277
(2011) · Cited by 1,192
Hovy, E., Marcus, M., Palmer, M., Ramshaw, L., & Weischedel, R. (2006) · Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers on XX - NAACL '06 · doi:10.3115/1614049.1614064 · Cited by 806
Learning Accurate, Compact, and Interpretable Tree Annotation
Petrov, S., Barrett, L., Thibaux, R., & Klein, D. (2006) · Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the ACL - ACL '06 · doi:10.3115/1220175.1220230 · Cited by 793
Self-citation
Research works printed but not linked: 5
- [2][2] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 , 2014. - · arXiv
- [4][4] S´ ebastien Jean, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. On using very large target vocabulary for neural machine translation. arXiv preprint arXiv:1412.2007 , 2014. - · arXiv
- [24][24] Ronan Collobert. Deep learning for efficient discriminative parsing. In International Conference on Artificial Intelligence and Statistics , 2011. -
- [31][31] Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. Show and tell: A neural image caption generator. arXiv preprint arXiv:1411.4555 , 2014. - · arXiv
- [32][32] Zoubin Ghahramani. A neural network for learning how to parse tree adjoining grammar. B.S.Eng Thesis, University of Pennsylvania, 1990.
Cited by
Sorted by citations
Items 401–464 of 464
Transforming thermal comfort model and control in the tropics : a machine-learning approach
Hu, W. (2020) · doi:10.32657/10356/141578 · Cited by 2
Transforming the Generative Pretrained Transformer Into Augmented Business Text WriterPreprint
Khalil, F., & Pipa, G. (2021) · Research Square · doi:10.21203/rs.3.rs-1170589/v1 · Cited by 2
A comparative study of deep learning algorithms for hate speech detection on Twitter
Mutanga, R. T. (2021) · doi:10.51415/10321/3776 · Cited by 2
McCarthy, A. D., Zhang, H., Kumar, S., Stahlberg, F., & Wu, K. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.19 · Cited by 2
Named Entity Recognition Through Learning from Experts
Andrews, M. (2015) · Proceedings in adaptation, learning and optimization · doi:10.1007/978-3-319-27000-5_23 · Cited by 1
Adaptive Importance Sampling from Finite State Automata
Teichmann, C., Wansing, K., & Koller, A. (2016) · doi:10.18653/v1/w16-2402 · Cited by 1
Allaparthi, S. T., Yaparla, G., & Pudi, V. (2018) · IEEE International Conference on Big Knowledge (ICBK) · doi:10.1109/icbk.2018.00017 · Cited by 1
A Rich-label Constituency Tree for Constituency Parsing
Hu, Y., & Fenglu, Z. (2019) · IEEE Joint International Information Technology and Artificial Intelligence Conference (ITAIC) · doi:10.1109/itaic.2019.8785583 · Cited by 1
Yli-Jyrä, A. (2019) · Journal of Language Modelling · doi:10.15398/jlm.v7i2.213 · Cited by 2
Title Generation and Keyphrase Extraction from Persian Scientific Texts
Mohseni, M., & Faili, H. (2020) · International Computer Conference, Computer Society of Iran (CSICC) · doi:10.1109/csicc49403.2020.9050113 · Cited by 1
Storytelling AI: A Generative Approach to Story Narration.
Fotedar, S., Vannisselroij, K., Khalil, S. S., & Ploeg, B. (2020) · International Joint Conference on Artificial Intelligence · Cited by 1
Sharma, M. K., Jain, M., Garg, M., & Tripathi, M. M. (2021) · International Conference on Innovative Practices in Technology and Management (ICIPTM) · doi:10.1109/iciptm52218.2021.9388359 · Cited by 1
Análise Semântica com base em AMR para o Português
Anchiêta, R. T., & Pardo, T. A. S. (2022) · Linguamática · doi:10.21814/lm.14.1.358 · Cited by 1
A Unified Framework for Synaesthesia Analysis
Sheng, K., Wang, Z., Zhao, Q., Jiang, X., & Zhou, G. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.401 · Cited by 1
A New Multimodal Large Model Framework for Knowledge-enhanced Image Caption Generation
Wan, T., Liu, H., & Geng, L. (2024) · IEEE International Conference on Advances in Electrical Engineering and Computer Applications (AEECA) · doi:10.1109/aeeca62331.2024.00102 · Cited by 1
Neural machine translation with constraints
Zhang, B., Wang, X., Xiong, D., & Li, J. (2018) · Scientia Sinica Informationis · doi:10.1360/n112017-00222 · Cited by 1
Nonlinearized Relevance Propagation
Zhang, Q., & Ohsawa, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-97304-3_69 · Cited by 1
Dynamic Delay Based Cyclic Gradient Update Method for Distributed Training
Hu, W., Wang, P., Wang, Q., Zhou, Z., Xiang, H. Y., Li, M., & Shi, Z. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-03338-5_46 · Cited by 1
Robot Vision to Audio Description Based on Deep Learning for Effective Human-Robot Interaction
Park, D., Kang, K.-M., Bae, J.-W., & Han, J.-H. (2019) · The Journal of Korea Robotics Society · doi:10.7746/jkros.2019.14.1.022 · Cited by 1
EmoAtt at EmoInt-2017: Inner attention sentence embedding for Emotion Intensity
Marrese-Taylor, E., & Matsuo, Y. (2017) · Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA) · doi:10.18653/v1/w17-5232 · Cited by 1
Attention Based Speech Model for Japanese Recognization
Mu, D., Zhu, T., Xu, G., Li, H., Guo, D., & Liu, Y. (2019) · IEEE International Conference on Smart Internet of Things (SmartIoT) · doi:10.1109/smartiot.2019.00071 · Cited by 1
Crossmodal attentive skill learner: learning in Atari and beyond with audio–video inputs
Kim, D. K., Omidshafiei, S., Pazis, J., & How, J. P. (2020) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-019-09439-5 · Cited by 1
Neural Combinatory Constituency Parsing
Chen, Z., Zhang, L., Imankulova, A., & Komachi, M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.194 · Cited by 1
Study the Quantum Transport Process: Machine Learning Simulates Quantum Conditional Master Equation
Hu, Y., Li, X., & Zhu, Q.-S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-78609-0_12 · Cited by 1
Entity-Aware Biaffine Attention for Constituent Parsing
Bai, X., Yin, N., Zhang, X., Wang, X., & Luo, Z. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86362-3_16 · Cited by 1
Online DVFS using Deep Learning: Sequence to Sequence LSTM Networks with AttentionPreprint
Thethi, S. K., & Kumar, R. (2022) · doi:10.36227/techrxiv.21539991 · Cited by 1
Advanced Language Understanding with Syntax-Enhanced TransformerPreprint
Rine, W., Patel, R., & Steve, N. (2023) · Preprints.org · doi:10.20944/preprints202312.1673.v1 · Cited by 1
Abstractive Text Summarization for Bangla Language Using NLP and Machine Learning Approaches
Miazee, A. A., Roy, T., Islam, M. R., & Safat, Y. (2025) · International Conference on Electrical, Computer and Communication Engineering · doi:10.1109/ecce64574.2025.11013982 · Cited by 1
AGI and ReflexivityPreprint
Faudemay, P. (2016) · arXiv (Cornell University) · doi:10.13140/rg.2.1.1888.5520
Structured Prediction Theory and Voted Risk Minimization.Preprint
Cortes, C., Mohri, M., Kuznetsov, V., & Yang, S. C. (2016) · arXiv (Cornell University)
RStudio AI Blog: Attention-based Neural Machine Translation with Keras
Keydana, S. (2018)
Wang, W. (2018) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/75872
Use Generalized Representations, But Do Not Forget Surface Features
Moosavi, N. S., & Strube, M. (2017) · White Rose Research Online (University of Leeds, The University of Sheffield, University of York) · doi:10.48550/arxiv.1702.07507
On Multilingual Training of Neural Dependency ParsersPreprint
Zapotoczny, M., Rychlikowski, P., & Chorowski, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.10209
Neural Multi-Source Morphological ReinflectionPreprint
Kann, K., Cotterell, R., & Schütze, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.06027
TensorFlow for R: Attention-based Neural Machine Translation with Keras
Keydana, S. (2018)
Wen, W., Wang, Z., Liu, J., Chen, C., & Li, N. (2019) · International Symposium on Computer Science and Intelligent Control (ISCSIC) · doi:10.1145/3386164.3389101
RStudio AI Blog: torch time series, final episode: Attention
Keydana, S. (2021)
Li, C., Zhang, Q., Zhao, Z., Li, G., & Schuller, B. W. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412577
The Limitations of Limited Context for Constituency Parsing
Li, Y., & Risteski, A. (2021) · doi:10.18653/v1/2021.acl-long.208 · Cited by 6
Structured pointing networks for natural language understanding
Nguyen, T. T. (2021) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/152207
Unsupervised paraphrasing via syntactic template sampling
Bao, Y., 黄, 书., 周, 浩., 李, 磊., 戴, 新., & 陈, 家. (2021) · Scientia Sinica Informationis · doi:10.1360/ssi-2021-0065
Network Embedding with Dual Generation Tasks
Li, N., Liu, J., He, Z., Zhang, C., & Xie, J. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3187851 · Cited by 1
Online DVFS using Deep Learning: Sequence to Sequence LSTM Networks with AttentionPreprint
Thethi, S. K., & Kumar, R. (2022) · doi:10.36227/techrxiv.21539991.v1
Dependency as Modality, Parsing as Permutation
Kogkalidis, K. (2023) · Utrecht University Repository (Utrecht University) · doi:10.33540/1721
Truong, N. T., Nguyen, C. T., Zanibbi, R., Mouchère, H., & Nakagawa, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4504151
Towards English-centric Zero-shot Neural Machine Translation: The Analysis and Solution
Wang, S. (2023) · IEEE International Conference on Sensors, Electronics and Computer Engineering (ICSECE) · doi:10.1109/icsece58870.2023.10263323
Generating Diverse Descriptions from Semantic Graphs
Han, J., Beck, D., & Cohn, T. (2021) · International Conference on Natural Language Generation · doi:10.18653/v1/2021.inlg-1.1
Retrieval-Augmented Parsing for Complex Graphs by Exploiting Structure and Uncertainty
Zi, L., Yuan, Q., Pasupat, P., Liu, J. Z., & Shang, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.419
Pushdown Layers: Encoding Recursive Structure in Transformer Language Models
Murty, S., Sharma, P., Andreas, J., & Manning, C. D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.195 · Cited by 8
Approximating CKY with Transformers
Khalighinejad, G., Liu, O., & Wiseman, S. M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.934
VietLegalQA: Unsupervised Legal Question Answering for Vietnamese Using Cloze Translation Approach
Ngoc, K. N., Trong, P. N., Minh, C. L. N., Khang, N. H. G., Quoc, T. N., & Hoang, V. T. (2023) · International Conference on Computational Intelligence and Communication Networks (CICN) · doi:10.1109/cicn59264.2023.10402152
On the Challenges of Fully Incremental Neural Dependency Parsing
Ezquerro, A., Gómez‐Rodríguez, C., & Vilares, D. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-short.7
Lee, G., & Lee, B.-M. (2024) · The Journal of AsiaTEFL · doi:10.18823/asiatefl.2024.21.2.1.270
Utilizing Deep Learning Technologies for Innovative Application
Sikand, R., Gautam, J., Husain, S. O., Sapaev, I. B., Badhoutiya, A., & Joshi, A. (2024) · International Conference on Communication, Computing and Energy Efficient Technologies (I3CEET) · doi:10.1109/i3ceet61722.2024.10993592
Gille, C. (2025) · Springer eBooks · doi:10.1007/978-3-031-84300-6_12
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
“In contrast to RNN sequence-to-sequence models [37], the Transformer outperforms the BerkeleyParser [29] even when training only on the WSJ training set of 40K sentences.”
BackgroundMethodInfluentialSelf-citation
Metadata
- Authors
- Vinyals, O., Kaiser, L., Koo, T., Petrov, S., Sutskever, I., & Hinton, G.
- Affiliations
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2014
- Volume
- 28
- Pages
- 2773-2781
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1412.7449
- OpenAlex
- W1869752048
- Cited by
- 464
- References
- 31
- Citations by year
- 2025: 2 · 2024: 11 · 2023: 26 · 2022: 24 · 2021: 42 · 2020: 42 · 2019: 71 · 2018: 88 · 2017: 72 · 2016: 57 · 2015: 29
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- neural information processing systems
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
- Keywords
- synthetic training data, data efficiency, natural language parsing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
