Compound Probabilistic Context-Free Grammars for Grammar Induction
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
We study a formalization of the grammar induction problem that models sentences as being generated by a compound probabilistic context free grammar.In contrast to traditional formulations which learn a single stochastic grammar, our context-free rule probabilities are modulated by a per-sentence continuous latent variable, which induces marginal dependencies beyond the traditional context-free assumptions.Inference in this grammar is performed by collapsed variational inference, in which an amortized variational posterior is placed on the continuous variable, and the latent trees are marginalized with dynamic programming.Experiments on English and Chinese show the effectiveness of our approach compared to recent state-of-the-art methods for grammar induction from words with neural language models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Statistical Decision Theory and Bayesian Analysis
Berger, J. O. (1985) · Springer Series in Statistics · doi:10.1007/978-1-4757-4286-2 · Cited by 7,722
Building a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553380 · Cited by 5,366 · cited 2×
“We employ a curriculum learning strategy (Bengio et al., 2009) where we train only on sentences of length up to 30 in the first epoch, and increase this length limit by 1 each epoch.”
MethodImproved Semantic Representations From Tree-Structured Long Short-Term Memory Networks
Tai, K. S., Socher, R., & Manning, C. D. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1150 · Cited by 3,081
Character-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D., & Rush, A. M. (2015) · arXiv · doi:10.48550/arxiv.1508.06615 · Cited by 1,530
Self-citationLearning Accurate, Compact, and Interpretable Tree Annotation
Petrov, S., Barrett, L., Thibaux, R., & Klein, D. (2006) · Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the ACL - ACL '06 · doi:10.3115/1220175.1220230 · Cited by 793 · cited 1×
“Our work is also related to latent variable PCFGs (Matsuzaki et al., 2005; Petrov et al., 2006; Cohen et al., 2012), which extend PCFGs to the latent variable setting by splitting nonterminal symbols into latent subsymbols.”
BackgroundConstituency Parsing with a Self-Attentive Encoder
Kitaev, N., & Klein, D. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1249 · Cited by 691
Recurrent Neural Network Grammars
Dyer, C., Kuncoro, A., Ballesteros, M., & Smith, N. A. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1024 · Cited by 517 · cited 3×
“This new representation is shifted onto the stack via the stack LSTM. Note that this RNNG parameterization is slightly different than the original from Dyer et al. (2016), which does not ignore constituent labels and utilizes a bidirectional LSTM as the composition function instead of a tree LSTM.”
MethodSelf-citationTargeted Syntactic Evaluation of Language Models
Marvin, R., & Linzen, T. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1151 · Cited by 334 · cited 1×
“For grammaticality judgment, we modify the publicly available dataset from Marvin and Linzen (2018) 23 to only keep sentence pairs that did not have any unknown words with respect to our PTB vocabulary of 10K words.”
Method
Cited by
Meta-Sim2: Unsupervised Learning of Scene Structure for Synthetic Data Generation
Devaranjan, J., Kar, A., & Fidler, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58520-4_42 · Cited by 73
End-to-end learning of multiple sequence alignments with differentiable Smith–Waterman
Petti, S., Bhattacharya, N., Rao, R., Dauparas, J., Thomas, N., Zhou, J., Rush, A. M., Koo, P., & Ovchinnikov, S. (2023) · Bioinformatics · doi:10.1093/bioinformatics/btac724 · Cited by 54
Self-citationShen, Y., Tay, Y., Zheng, C., Bahri, D., Metzler, D., & Courville, A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.559 · Cited by 29
“…49.5 URNNG (Kim et al., 2019b) 52.4 C-PCFG (Kim et al., 2019a) 55.2 Neural L-PCFGs (Zhu et al., 2020) 55.31 StructFormer 54.0 (0.3) the C-PCFG (Kim et al., 2019a) achieve a stronger parsing performance with its strong linguistic constraints (e.g. a finite set of production rules),…”
BackgroundMethodInfluentialUncertainty in Natural Language Generation: From Theory to Applications
Baan, J., Daheim, N., Ilia, E., Ulmer, D., Li, H.-S., Fernández, R., Plank, B., Sennrich, R., Zerva, C., & Aziz, W. (2023) · arXiv · doi:10.48550/arxiv.2307.15703 · Cited by 6
A Theory of Emergent In-Context Learning as Implicit Structure Induction
Hahn, M., & Goyal, N. (2023) · arXiv · doi:10.48550/arxiv.2303.07971
Metadata
- Authors
- Kim, Y., Dyer, C., & Rush, A.
- Venue
- Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
- Published
- 2019
- Pages
- 2369-2385
- OpenAlex
- W2949399644
- Cited by
- 117
- References
- 10
- FWCI
- 9.42
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Compound Probabilistic Context-Free Grammars for Grammar Induction. https://app.cochat.ai/papers/10.18653/v1/p19-1228
- The paper
- Kim, Y., Dyer, C., & Rush, A. (2019). Compound Probabilistic Context-Free Grammars for Grammar Induction. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. https://doi.org/10.18653/v1/p19-1228
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.