The DOI matches its DataCite record; 38 of the 38 references it cites were found in a registry or an index.
Template-Based Named Entity Recognition Using BART
Open accessAbstract
There is a recent interest in investigating few-shot NER, where the low-resource target domain has different label sets compared with a resource-rich source domain. Existing methods use a similarity-based metric. However, they cannot make full use of knowledge transfer in NER model parameters. To address the issue, we propose a template-based method for NER, treating NER as a language model ranking problem in a sequence-to-sequence framework, where original sentences and statement templates filled by candidate named entity span are regarded as the source sequence and the target sequence, respectively. For inference, the model is required to classify each candidate span based on the corresponding template scores. Our experiments demonstrate that the proposed method achieves 92.55% F1 score on the CoNLL03 (rich-resource task), and significantly better than fine-tuning BERT 10.88%, 15.34%, and 11.73% F1 score on the MIT Movie, the MIT Restaurant, and the ATIS (low-resource task), respectively.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Cui et al. [17] manually design lists such as”
From Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2023) · cites this paper as [17]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Token or text-span answer spaces are widely used in classification tasks (e.g., sentiment classification; Yin et al. [144]) as well as in other tasks such as relation extraction [100] or named entity recognition [17].”
From Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2023) · cites this paper as [17]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 3×
“The current dominant methods use a sequential neural network such as BiLSTM (Hochreiter and Schmidhuber, 1997) and BERT (Devlin et al., 2019) is used to represent the input text, and softmax (Chiu and Nichols, 2016; Strubell et al., 2017; Cui and Zhang, 2019) or CRF (Lample et al., 2016; Ma and…”
MethodJakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 5×
“We use BERT (Devlin et al., 2019) and BART (Lewis et al.”
MethodInfluential(2008) · Journal of Machine Learning Research · Cited by 33,051
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · cited 3×
“We use BERT (Devlin et al., 2019) and BART (Lewis et al., 2020) as our E NCODER to learn the sequence representation.”
MethodResultNeural Architectures for Named Entity Recognition
Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., & Dyer, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1030 · Cited by 4,401 · cited 6×
“Third, compared with traditional meth-ods (pre-trained model with a softmax/CRF), our method can be applied to arbitrary new categories of named entities without changing the output layer, and therefore allows continual learning (Lin et al., 2020).”
BackgroundMethodInfluentialIntroduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition
Tjong Kim Sang, E. F., & De Meulder, F. (2003) · Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003 - · doi:10.3115/1119176.1119195 · Cited by 2,788 · cited 2×
“Named entity recognition (NER) is a fundamental task in natural language processing, which identi-fies mention spans from text inputs according to pre-defined entity categories (Tjong Kim Sang and De Meulder, 2003), such as location, person, organization, etc.”
BackgroundMethodEnd-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
Ma, X., & Hovy, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1101 · Cited by 2,567 · cited 6×
“Third, compared with traditional meth-ods (pre-trained model with a softmax/CRF), our method can be applied to arbitrary new categories of named entities without changing the output layer, and therefore allows continual learning (Lin et al., 2020).”
BackgroundMethodInfluentialNamed Entity Recognition with Bidirectional LSTM-CNNs
Chiu, J., & Nichols, E. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00104 · Cited by 2,007 · cited 4×
“, 2019) is used to represent the input text, and softmax (Chiu and Nichols, 2016; Strubell et al., 2017; Cui and Zhang, 2019) or CRF (Lample et al.”
MethodInfluentialLanguage Models as Knowledge Bases?
Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1250 · Cited by 1,834 · cited 1×
“Petroni et al. (2019) extract relation between entities from BERT by constructing cloze-style templates.”
MethodSelf-citationIntroduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity RecognitionPreprint
Sang, E. T. K. (2002) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0209010 · Cited by 1,677
Making Pre-trained Language Models Better Few-shot Learners
Gao, T., Fisch, A., & Chen, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.295 · Cited by 1,380 · cited 3×
“First, due to the good generalization ability of pre-trained models (Brown et al., 2020; Gao et al., 2020), the network can effectively leverage labeled instances in the new domain for tine-tuning.”
BackgroundExploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference
Schick, T., & Schütze, H. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.20 · Cited by 1,305 · cited 2×
“Schick and Schütze (2020) rephrase inputs as cloze questions for text classification.”
BackgroundLUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention
Yamada, I., Asai, A., Shindo, H., Takeda, H., & Matsumoto, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.523 · Cited by 739
Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence
Sun, C. C., Huang, L., & Qiu, X. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1035 · Cited by 728 · cited 1×
“Sun et al. (2019) use templates to construct auxiliary sentences, and transform aspect sentiment task as a sentence-pair classification task.”
MethodFast and Accurate Entity Recognition with Iterated Dilated Convolutions
Strubell, E., Verga, P. W., Belanger, D., & McCallum, A. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1283 · Cited by 532 · cited 4×
“…BiLSTM (Hochreiter and Schmidhuber, 1997) and BERT (Devlin et al., 2019) is used to represent the input text, and softmax (Chiu and Nichols, 2016; Strubell et al., 2017; Cui and Zhang, 2019) or CRF (Lample et al., 2016; Ma and Hovy, 2016; Luo et al., 2020) output layers to assign named entity…”
MethodInfluentialMulti-Domain Joint Semantic Frame Parsing Using Bi-Directional RNN-LSTM
Hakkani-Tür, D. Z., Tür, G., Çelikyılmaz, A., Chen, Y.-N., Gao, J., Deng, L., & Wang, Y. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-402 · Cited by 476
Pooled Contextualized Embeddings for Named Entity Recognition
Akbik, A., Bergmann, T., & Vollgraf, R. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1078 · Cited by 292
Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning
Yang, Y., & Katiyar, A. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.516 · Cited by 242
Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification
Schick, T., Schmid, H., & Schütze, H. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.488 · Cited by 184
Design Challenges and Misconceptions in Neural Sequence LabelingPreprint
Yang, J., Liang, S., & Zhang, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.04470 · Cited by 145
Self-citationHierarchical Contextualized Representation for Named Entity Recognition
Luo, Y., Xiao, F., & Zhao, H. Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6363 · Cited by 142 · cited 6×
“…to represent the input text, and softmax (Chiu and Nichols, 2016; Strubell et al., 2017; Cui and Zhang, 2019) or CRF (Lample et al., 2016; Ma and Hovy, 2016; Luo et al., 2020) output layers to assign named entity tags (e.g. organization, person and location) or non-entity tags on each input token.”
BackgroundInfluentialA Local Detection Approach for Named Entity Recognition and Mention Detection
Xu, M., Jiang, H., & Watcharawittayakul, S. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1114 · Cited by 140 · cited 2×
“Ziyadi et al. (2020) follow a two-step approach (Lin et al., 2019; Xu et al., 2017), which first detects spans boundary and then recognizes entity types by comparing the similarity with the labeled instance.”
BackgroundMaking Pre-trained Language Models Better Few-shot Learners
Gao, T., Fisch, A., & Chen, D. (2020) · arXiv · doi:10.48550/arxiv.2012.15723 · Cited by 124 · cited 3×
“First, due to the good generalization ability of pre-trained models (Brown et al., 2020; Gao et al., 2020), the network can effectively leverage labeled instances in the new domain for tine-tuning.”
BackgroundSequence-to-Nuggets: Nested Entity Mention Detection via Anchor-Region Networks
Lin, H., Lu, Y., Han, X., & Sun, L. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1511 · Cited by 121 · cited 2×
“Ziyadi et al. (2020) follow a two-step approach (Lin et al., 2019; Xu et al., 2017), which first detects spans boundary and then recognizes entity types by comparing the similarity with the labeled instance.”
BackgroundHierarchically-Refined Label Attention Network for Sequence Labeling
Cui, L., & Zhang, Y. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1422 · Cited by 110 · cited 1×
“, 2019) is used to represent the input text, and softmax (Chiu and Nichols, 2016; Strubell et al., 2017; Cui and Zhang, 2019) or CRF (Lample et al.”
MethodSelf-citationAsgard: A portable architecture for multilingual dialogue systems
Liu, J., Pasupat, P., Cyphers, S., & Glass, J. R. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6639301 · Cited by 106 · cited 3×
“(2020), we use MIT Movie Review (Liu et al., 2013), MIT Restaurant Review (Liu et al.”
BackgroundMethodSelf-citationGCDT: A Global Context Enhanced Deep Transition Architecture for Sequence Labeling
Liu, Y., Meng, F., Zhang, J., Xu, J., Chen, Y., & Zhou, J. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1233 · Cited by 104 · cited 1×
“Some methods (Chiu and Nichols, 2016; Strubell et al., 2017) treat NER as a local classification problem at each input token, while other methods use CRF (Ma and Hovy, 2016) or a sequence-to-sequence framework (Zhang et al., 2018; Liu et al., 2019).”
MethodFew-Shot Named Entity Recognition: A Comprehensive StudyPreprint
Huang, J., Li, C., Subudhi, K., Jose, D., Balakrishnan, S., Chen, W., Peng, B., Gao, J., & Han, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.14978 · Cited by 75 · cited 6×
“For example, the performance of Huang et al. (2020) increases only 1.9% F1 score when the number of instances per entity type increase from 10 to 500.”
BackgroundMethodInfluentialLabel-Agnostic Sequence Labeling by Copying Nearest Neighbors
Wiseman, S., & Stratos, K. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1533 · Cited by 56 · cited 5×
“Compared with the distance-based method (Wiseman and Stratos, 2019; Ziyadi et al., 2020; Huang et al., 2020), our method shows more improvement when the number of target-domain labeled data increases, because the distance-based method just optimizes its searching threshold rather than updating its…”
BackgroundMethodInfluentialExample-Based Named Entity RecognitionPreprint
Ziyadi, M., Sun, Y., Goswami, A., Huang, J., & Chen, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.10570 · Cited by 37 · cited 7×
“For example, when there are only 10 training instances, the method of Ziyadi et al. (2020) gives a F1 score of 40.1% on MIT Movie, as compared to 28.3% by BERT, despite that BERT requires re-training with a different output layer on both CoNLL03 and MIT Movie.”
BackgroundMethodInfluentialNamed Entity Recognition - Is There a Glass Ceiling?
Ziembicki, D., Biecek, P., Stanisławek, T., Wróblewska, A., & Wójcicka, A. (2019) · ReIn UW (University of Warsaw) · doi:10.18653/v1/k19-1058 · Cited by 28
Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification
Schick, T., Schmid, H., & Schütze, H. (2020) · Open access LMU (Ludwid Maxmilian's Universitat Munchen) · doi:10.5282/ubm/epub.74034 · Cited by 26 · cited 1×
“Schick et al. (2020) and Gao et al. (2020) extend Schick and Schütze (2020) by automatically generating label words and templates, respectively.”
BackgroundLearning Tag Dependencies for Sequence Tagging
Zhang, Y., Chen, H., Zhao, Y., Liu, Q., & Yin, D. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/637 · Cited by 26 · cited 1×
“Some methods (Chiu and Nichols, 2016; Strubell et al., 2017) treat NER as a local classification problem at each input token, while other methods use CRF (Ma and Hovy, 2016) or a sequence-to-sequence framework (Zhang et al., 2018; Liu et al., 2019).”
MethodSelf-citationUncertainty-Aware Label Refinement for Sequence Labeling
Gui, T., Ye, J., Zhang, Q., Li, Z., Fei, Z., Gong, Y., & Huang, X. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.181 · Cited by 22
Ji, Y., & Gao, S. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 10
Lin, S., Gao, J., Zhang, S., et al. (2020) · World Wide Web · doi:10.1007/s11280-019-00758-x · Cited by 6 · cited 4×
“Third, compared with traditional meth-ods (pre-trained model with a softmax/CRF), our method can be applied to arbitrary new categories of named entities without changing the output layer, and therefore allows continual learning (Lin et al., 2020).”
BackgroundMethodInfluential
Research works printed but not linked: 1
- Jie Yang, Shuailong Liang, and Yue Zhang. 2018. De- sign challenges and misconceptions in neural se- quence labeling. In Proceedings of the 27th Inter- national Conference on Computational Linguistics, pages 3879-3889, Santa Fe, New Mexico, USA. As- sociation for Computational Linguistics.
Cited by
Sorted by citations
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361
Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey
Min, B., Ross, H., Sulem, E., Veyseh, A. P. B., Nguyen, T. H., Sainz, O., Agirre, E., Heintz, I., & Roth, D. (2024) · ACM Computing Surveys · doi:10.1145/3605943 · Cited by 1,387
“[26] explore generation-based ranking for NER, especially in few-shot and cross-domain few-shot settings.”
BackgroundInfluentialA Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities
Song, Y., Wang, T., Cai, P., Mondal, S. K., & Sahoo, J. P. (2023) · ACM Computing Surveys · doi:10.1145/3582688 · Cited by 721
“Different tasks [107] could have different blanks positions to test whether the model has learned the corresponding semantic knowledge [107, 111].”
BackgroundMethodUnified Structure Generation for Universal Information Extraction
Lu, Y., Liu, Q., Dai, D., Xiao, X., Lin, H., Han, X., Sun, L., & Wu, H. R. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.395 · Cited by 520
Large language models are few-shot clinical information extractors
Agrawal, M., Hegselmann, S., Lang, H., Kim, Y., & Sontag, D. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.130 · Cited by 358
“For example, given a candidate entity X and full model access, the entity type can be predicted via an argmax over the possible types Y of the probability of statements like “X is a Y entity” (Cui et al., 2021).”
Background
Metadata
- Authors
- Cui, L., Wu, Y., Liu, J., Yang, S., & Zhang, Y.
- Affiliations
- Westlake University (CN)
- Zhejiang University (CN)
- Corresponding author
- Zhang, Yue
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Pages
- 1835-1845
- arXiv
- 2106.01760
- OpenAlex
- W3176489198
- Semantic Scholar
- ce498651107588db67adcfbb5479bdb416f4de2f
- Cited by
- 342
- References
- 38
- Citations by year
- 2026: 16 · 2025: 65 · 2024: 80 · 2023: 121 · 2022: 56 · 2021: 4
- FWCI
- 30.09
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Findings of the Association for Computational Linguistics: ACL (Findings of ACL) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (61976180)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- named entity recognition, few-shot learning, BART, Seq2Seq, low-resource domains, knowledge transfer, domain adaptation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
