CoChat
Sign Up
Cited by 13Natural Language Processing TechniquesTopic ModelingSyntax, Semantics, Linguistic Variation

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

Potential and Limitations of LLMs in Capturing Structured Semantics: A Case Study on SRL

Open access
Cheng, Ning, Yan, Zhaohui, Wang, Ziming, Li, Zhijie, Yu, Jiaming, Zheng, Zilong, Tu, Kewei, Xu, Jinan, Han, Wenjuan·2024·Lecture Notes in Computer Science
DOI 10.1007/978-981-97-5663-6_5
Open this paper in CoChat — free accountOpen PDF

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/978-981-97-5663-6_5

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • LSTM-based models demonstrate increased errors when sequential and structural information conflict.

    “LSTM-based models demonstrate increased errors when sequential and structural information conflict”

    From A Survey of Context Engineering for Large Language Models (2025) · cites this paper as [167]

    Section 2.2 discusses LSTM-based models only as the subject of a study of syntax-sensitive dependencies; it does not report increased errors when sequential and structural information conflict. A search for that conflict and its effect on errors found no passage in our copy that addresses the claim. (LLM, conf=0.75)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 25

  1. Wei, J., et al.: Chain-of-thought prompting elicits reasoning in large language models. Adv. Neural. Inf. Process. Syst. 35, 24824–24837 (2022)
  2. Wei, X., Chen, Y., Cheng, N., Cui, X., Xu, J., Han, W.: CollabKG: a Learnable human-machine-cooperative information extraction toolkit for (Event) knowledge graph construction. arXiv preprint arXiv:2307.00769 (2023) · arXiv
  3. Jiang, G., Xu, M., Zhu, S.-C., Han, W., Zhang, C., Zhu, Y.: Evaluating and inducing personality in pre-trained language models. Adv. Neural Inform. Process. Syst. 36 (2024)
  4. Shin, R., Van Durme, B.: Few-shot semantic parsing with language models trained on code. In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5417–5425 (2022) · DOI
  5. He, H., Choi, J.: Establishing strong baselines for the new decade: sequence tagging, syntactic and semantic parsing with BERT. In: The Thirty-Third International Flairs Conference (2020)
  6. Tenney, I., et al.: What do you learn from context? probing for sentence structure in contextualized word representations. arXiv preprint arXiv:1905.06316 (2019) · arXiv
  7. Vilares, D., Strzyz, M., Søgaard, A., Gómez-Rodríguez, C.: Parsing as pretraining. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 9114–9121 (2020) · DOI
  8. Fitzgerald, N., Michael, J., He, L., Zettlemoyer, L.: Large-scale QA-SRL parsing. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2051–2060 (2018) · DOI
  9. Swier, R.S., Stevenson, S.: Unsupervised semantic role labellin. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pp. 95–102 (2004)
  10. Titov, I., Khoddam, E.: Unsupervised induction of semantic roles within a reconstruction-error minimization framework. In: Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pp. 1–10. Association for Computational Linguistics, (2015) · DOI
  11. Titov, I., Klementiev, A.: A Bayesian approach to unsupervised semantic role induction. In: Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pp. 12–22 (2012)
  12. Abend, O., Reichart, R., Rappoport, A.: Unsupervised argument identification for semantic role labeling. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pp. 28–36 (2009) · DOI
  13. Drozdov, A., et al.: Compositional semantic parsing with large language models. In: The Eleventh International Conference on Learning Representations (2022)
  14. Clark, K., Khandelwal, U., Levy, O., Manning, C.D.: What does BERT look at? An analysis of BERT’s attention. In: Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp. 276–286 (2019) · DOI
  15. Linzen, T., Dupoux, E., Goldberg, Y.: Assessing the ability of LSTMs to learn syntax-sensitive dependencies. Trans. Assoc. Comput. Linguist. 4, 521–535 (2016) · DOI
  16. Melis, G., Dyer, C., Blunsom, P.: On the state of the art of evaluation in neural language models. In: International Conference on Learning Representations (2017)
  17. Wei, X., et al.: Zero-shot information extraction via chatting with chatgpt. arXiv preprint arXiv:2302.10205 (2023) · arXiv
  18. Kingsbury, P., Palmer, M.: Propbank: the next level of treebank. In: Proceedings of Treebanks and lexical Theories. Citeseer (2003)
  19. Carreras, X., Màrquez, L.: Introduction to the CoNLL-2005 shared task: semantic role labeling. In: Proceedings of the ninth conference on computational natural language learning (CoNLL-2005), pp. 152–164 (2005) · DOI
  20. Pradhan, S., Moschitti, A., Xue, N., Uryupina, O., Zhang, Y.: CoNLL-2012 shared task: modeling multilingual unrestricted coreference in OntoNotes. In: Joint conference on EMNLP and CoNLL-shared task, pp. 1–40 (2012)
  21. Touvron, H., et al.: Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023) · arXiv
  22. Fei, H., Wu, S., Ren, Y., Li, F., Ji, D.: Better combine them together! integrating syntactic constituency and dependency representations for semantic role labeling. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 549–559 (2021) · DOI
  23. Zhang, Y., Xia, Q., Zhou, S., Jiang, Y., Fu, G., Zhang, M.: Semantic role labeling as dependency parsing: Exploring latent tree structures inside arguments. arXiv preprint arXiv:2110.06865 (2021) · arXiv
  24. Wang, N., et al.: An MRC framework for semantic role labeling. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 2188–2198 (2021)
  25. Fleiss, J.L.: Measuring nominal scale agreement among many raters. Psychol. Bull. 76, 378 (1971) · DOI · Paper page

Cited by

Sorted by citations

Metadata

Authors
Cheng, N., Yan, Z., Wang, Z., Li, Z., Yu, J., Zheng, Z., Tu, K., Xu, J., & Han, W.
Affiliations
  • Beijing Jiaotong University (CN)
  • ShanghaiTech University (CN)
  • Tsinghua University (CN)
  • University of Oxford (GB)
  • Beijing Institute for General Artificial Intelligence (CN)
Corresponding author
  • Cheng, Ning
  • Yan, Zhaohui
  • Han, Wenjuan
ORCID
Venue
Lecture Notes in Computer Science
Publisher
Springer Nature Singapore
Published
2024
Pages
50-61
ISSN
  • 0302-9743 (ISSN-L)
  • 1611-3349
DOI
10.1007/978-981-97-5663-6_5
OpenAlex
W4401235192
Cited by
13
References
5
Citations by year
2026: 3 · 2025: 10
FWCI
3.14
Type
conference paper
Language
en
Open access
Licence
Locations
  • Lecture notes in computer science (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Social Sciences › Arts and Humanities › Language and Linguistics › Syntax, Semantics, Linguistic Variation
Keywords
large language models, semantic role labeling, few-shot prompting, argument extraction, human error analysis, LLM interpretability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review