The Penn Treebank: annotating predicate argument structure
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
The Penn Treebank has recently implemented a new syntactic annotation scheme, designed to highlight aspects of predicate-argument structure. This paper discusses the implementation of crucial aspects of this new annotation scheme. It incorporates a more consistent treatment of a wide range of grammatical phenomena, provides a set of coindexed null elements in what can be thought of as "underlying" position for phenomena such as wh-movement, passive, and the subjects of infinitival constructions, provides some non-context free annotational mechanism to allow the structure of discontinuous constituents to be easily recovered, and allows for a clear, concise tagging system for some semantic roles.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Comprehensive Grammar of the English Language
Huddleston, R., Quirk, R., Greenbaum, S., Leech, G., & Svartvik, J. (1988) · Language · doi:10.2307/415437 · Cited by 7,587
Building a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Self-citationA Comprehensive Grammar of the English Language
Beauvais, P. J., Quirk, R., Greenbaum, S., Leech, G., & Svartvik, J. (1987) · College Composition and Communication · doi:10.2307/357731 · Cited by 5,488
FREQUENCY ANALYSIS OF ENGLISH USAGE: LEXICON AND GRAMMAR
Francis, W. N., Kučera, H., & Mackie, A. W. (1983) · Medical Entomology and Zoology · Cited by 2,310
Structural ambiguity and lexical relations
Hindle, D., & Rooth, M. (1990) · Proceedings of the workshop on Speech and Natural Language - HLT '90 · doi:10.3115/116580.116664 · Cited by 503
Procedure for quantitatively comparing the syntactic coverage of English grammars
Klavans, J., Liberman, M., Marcus, M., Roukos, S., Santorini, B., Strzalkowski, T., Abney, S., Flickenger, S., Gdaniec, C., Grishman, C., et al. (1991) · Proceedings of the workshop on Speech and Natural Language - HLT '91 · doi:10.3115/112405.112467 · Cited by 432
Self-citationThe Computational Analysis of English: A Corpus-Based Approach
Francis, W. N., Garside, R., Leech, G., & Sampson, G. (1989) · Language · doi:10.2307/415358 · Cited by 216
The Computational Analysis of English: A Corpus-Based Approach
Chapelle, C., Garside, R., Leech, G., & Sampson, G. (1988) · TESOL Quarterly · doi:10.2307/3587270 · Cited by 211
Towards history-based grammars: using richer models for probabilistic parsing
Black, E. W., Jelinek, F., Lafferty, J., Magerman, D. M., Mercer, R. L., & Roukos, S. (1993) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/981574.981579 · Cited by 145
Magerman, D. M., & Marcus, M. P. (1991) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/977180.977184 · Cited by 72
Self-citation
Cited by
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Head-Driven Statistical Models for Natural Language Parsing
Collins, M. (2003) · Computational Linguistics · doi:10.1162/089120103322753356 · Cited by 1,826
AutoML: A survey of the state-of-the-art
He, X., Zhao, K., & Chu, X. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106622 · Cited by 1,691
Automatic Labeling of Semantic Roles
Gildea, D., & Jurafsky, D. (2002) · Computational Linguistics · doi:10.1162/089120102760275983 · Cited by 1,626
CoNLL-X shared task on multilingual dependency parsing
Buchholz, S., & Marsi, E. (2006) · Proceedings of the Tenth Conference on Computational Natural Language Learning - CoNLL-X '06 · doi:10.3115/1596276.1596305 · Cited by 963
Three generative, lexicalised models for statistical parsing
Collins, M. (1997) · Proceedings of the 35th annual meeting on Association for Computational Linguistics - · doi:10.3115/976909.979620 · Cited by 743
The Penn Chinese TreeBank: Phrase structure annotation of a large corpus
XUE, N., XIA, F., CHIOU, F.-D., & PALMER, M. (2005) · Natural Language Engineering · doi:10.1017/s135132490400364x · Cited by 666
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.04615 · Cited by 569
Algorithms for Deterministic Incremental Dependency Parsing
Nivre, J. (2008) · Computational Linguistics · doi:10.1162/coli.07-056-r1-07-027 · Cited by 435
The Penn Treebank: An Overview
Taylor, A., Marcus, M., & Santorini, B. (2003) · Text, Speech and Language Technology · doi:10.1007/978-94-010-0201-1_1 · Cited by 414
Self-citationThe Prague Dependency Treebank
Böhmová, A., Hajič, J., Hajičová, E., & Hladká, B. (2003) · Text, Speech and Language Technology · doi:10.1007/978-94-010-0201-1_7 · Cited by 409
CCGbank: A Corpus of CCG Derivations and Dependency Structures Extracted from the Penn Treebank
Hockenmaier, J., & Steedman, M. (2007) · Computational Linguistics · doi:10.1162/coli.2007.33.3.355 · Cited by 383
Frequency of basic English grammatical structures: A corpus analysis
Roland, D., Dick, F., & Elman, J. L. (2007) · Journal of Memory and Language · doi:10.1016/j.jml.2007.03.002 · Cited by 379
Generation that exploits corpus-based statistical knowledge
Langkilde, I., & Knight, K. (1998) · Proceedings of the 36th annual meeting on Association for Computational Linguistics - · doi:10.3115/980845.980963 · Cited by 348
A Computational Cognitive Model of Syntactic Priming
Reitter, D., Keller, F., & Moore, J. D. (2011) · Cognitive Science · doi:10.1111/j.1551-6709.2010.01165.x · Cited by 323
Probabilistic Inductive Logic Programming
De Raedt, L., & Kersting, K. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-78652-8_1 · Cited by 319
Representing Discourse Coherence: A Corpus-Based Study
Wolf, F., & Gibson, E. (2005) · Computational Linguistics · doi:10.1162/0891201054223977 · Cited by 303
An annotation scheme for free word order languages
Skut, W., Krenn, B., Brants, T., & Uszkoreit, H. (1997) · Proceedings of the fifth conference on Applied natural language processing - · doi:10.3115/974557.974571 · Cited by 301
Uncertainty About the Rest of the Sentence
Hale, J. (2006) · Cognitive Science · doi:10.1207/s15516709cog0000_64 · Cited by 296
Syntactic structure building in the anterior temporal lobe during natural story listening
Brennan, J., Nir, Y., Hasson, U., Malach, R., Heeger, D. J., & Pylkkänen, L. (2012) · Brain and Language · doi:10.1016/j.bandl.2010.04.002 · Cited by 279
“Syntactic structure was determined using an automated parser (Bikel, 2002), which had been trained on a corpus of text from the Wall Street Journal for which syntactic phrase-structure information was marked in accordance with the Treebank 2 scheme (Marcus et al., 1994).”
BackgroundMethodRiezler, S., King, T. H., Kaplan, R. M., Crouch, R., Maxwell, J. T., & Johnson, M. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073129 · Cited by 279
Support Vector Learning for Semantic Argument Classification
Pradhan, S., Hacioglu, K., Krugler, V., Ward, W., Martin, J. H., & Jurafsky, D. (2005) · Machine Learning · doi:10.1007/s10994-005-0912-2 · Cited by 275
Lessons from archives: strategies for collecting sociocultural data in machine learning
Jo, E. S., & Gebru, T. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372829 · Cited by 240
Contrasting Linguistic Patterns in Human and LLM-Generated News Text
Muñoz-Ortiz, A., Gómez-Rodríguez, C., & Vilares, D. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10903-2 · Cited by 94
“We use the annotation scheme of the English Penn Treebank, which has been widely used by the NLP community (Marcus et al. 1994).”
MethodText Segmentation by Cross Segment Attention
Lukasik, M., Dadachev, B., Papineni, K., & Simões, G. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.380 · Cited by 68
Foundations of Multimodal Co-learning
Zadeh, A., Liang, P. P., & Morency, L.-P. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.001 · Cited by 40
ARB-LLM: Alternating Refined Binarizations for Large Language Models
Li, Z., Yan, X., Zhang, T., Qin, H., Xie, D., Tian, J., shi, Z., Kong, L., Zhang, Y., & Yang, X. (2024) · arXiv · doi:10.48550/arxiv.2410.03129 · Cited by 1
“To evaluate the effectiveness of our proposed ARB-LLM X (ARB-X + CGB) and ARB-LLM RC (ARB-RC + CGB), we measure the perplexity of LLM’s outputs on WikiText2 (Merity et al., 2017), PTB (Marcus et al., 1994), as well as a part of the C4 (Raffel et al., 2020) data.”
MethodLittleBit: Ultra Low-Bit Quantization via Latent Factorization
Lee, B., Kim, D., You, Y., & Kim, Y. (2025) · arXiv · doi:10.48550/arxiv.2506.13771 · Cited by 7
“Additional results on the C4 [55] and PTB [56] datasets are provided in Appendix A.3.”
Metadata
- Authors
- Marcus, M., Kim, G., Marcinkiewicz, M. A., MacIntyre, R., Bies, A., Ferguson, M., Katz, K., & Schasberger, B.
- Venue
- Proceedings of the workshop on Human Language Technology - HLT '94
- Published
- 1994
- Pages
- 114
- OpenAlex
- W2093647425
- Cited by
- 742
- References
- 12
- FWCI
- 5.83
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.