Neural Architectures for Named Entity Recognition
Open accessAbstract
Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, Chris Dyer. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2016.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The LSTM-CRFs architecture has been widely adopted for named entity recognition tasks in both general and clinical domains.
“the long short-term memory-conditional random fields (LSTM-CRFs) architecture [25], which is a modified implementation of the recurrent neural network, has been widely adopted for named entity recognition (NER) tasks in both general and clinical domains.”
From Extracting Family History of Patients From Clinical Narratives: Exploring an End-to-End Solution With Deep Learning Models (2020) · cites this paper as [25]
The Introduction and Results sections describe an LSTM-CRF architecture for NER and report results in English, Dutch, German, and Spanish. They do not establish that it was widely adopted, and the paper does not report its use in clinical NER; the clinical-domain portion of the claim is unsupported. (LLM, conf=0.87)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · see citing works
ADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322 · see citing works
Flagged first, then by citations
Research works printed but not linked: 32
- Rie Kubota Ando and Tong Zhang. 2005a. A framework for learning predictive structures from multiple tasks and unlabeled data. The Journal of Machine Learning Research, 6:1817-1853.
- Rie Kubota Ando and Tong Zhang. 2005b. Learning predictive structures. JMLR, 6:1817-1853.
- Miguel Ballesteros, Chris Dyer, and Noah A. Smith. 2015. Improved transition-based dependency parsing by modeling characters instead of words with LSTMs. In Proceedings of EMNLP.
- Miguel Ballesteros, Yoav Golderg, Chris Dyer, and Noah A. Smith. 2016. Training with Explo- ration Improves a Greedy Stack-LSTM Parser. In arXiv:1603.03793. · arXiv
- Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gradient descent is difficult. Neural Networks, IEEE Transac- tions on, 5(2):157-166.
- Chris Callison-Burch, Philipp Koehn, Christof Monz, Kay Peterson, Mark Przybocki, and Omar F Zaidan. 2010. Findings of the 2010 joint workshop on sta- tistical machine translation and metrics for machine translation. In Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR, pages 17-53. Association for Computational Linguis- tics.
- Xavier Carreras, Lluís Màrquez, and Lluís Padró. 2002. Named entity extraction using adaboost, proceedings of the 6th conference on natural language learning. August, 31:1-4.
- Jason PC Chiu and Eric Nichols. 2015. Named en- tity recognition with bidirectional lstm-cnns. arXiv preprint arXiv:1511.08308. · arXiv
- Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12:2493- 2537.
- Silviu Cucerzan and David Yarowsky. 1999. Language independent named entity recognition combining mor- phological and contextual evidence. In Proceedings of the 1999 Joint SIGDAT Conference on EMNLP and VLC, pages 90-99.
- Silviu Cucerzan and David Yarowsky. 2002. Language independent ner using a unified model of internal and contextual evidence. In proceedings of the 6th confer- ence on Natural language learning-Volume 20, pages 1-4. Association for Computational Linguistics.
- Hong-Jie Dai, Po-Ting Lai, Yung-Chun Chang, and Richard Tzong-Han Tsai. 2015. Enhancing of chem- ical compound and drug name recognition using rep- resentative tag scheme and fine-grained tokenization. Journal of cheminformatics, 7(Suppl 1):S14.
- Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015. Transition- based dependency parsing with stack long short-term memory. In Proc. ACL.
- Jacob Eisenstein, Tae Yano, William W Cohen, Noah A Smith, and Eric P Xing. 2011. Structured databases of named entities from bayesian nonparametrics. In Proceedings of the First Workshop on Unsupervised Learning in NLP, pages 2-12. Association for Com- putational Linguistics.
- Radu Florian, Abe Ittycheriah, Hongyan Jing, and Tong Zhang. 2003. Named entity recognition through clas- sifier combination. In Proceedings of the seventh con- ference on Natural language learning at HLT-NAACL 2003-Volume 4, pages 168-171. Association for Com- putational Linguistics.
- Dan Gillick, Cliff Brunk, Oriol Vinyals, and Amarnag Subramanya. 2015. Multilingual language processing from bytes. arXiv preprint arXiv:1512.00103. · arXiv
- David Graff. 2011. Spanish gigaword third edition (ldc2011t12). Linguistic Data Consortium, Univer- sity of Pennsylvania, Philadelphia, PA.
- Alex Graves and Jürgen Schmidhuber. 2005. Framewise phoneme classification with bidirectional LSTM net- works. In Proc. IJCNN.
- Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. 2012. Improving neural networks by preventing co- adaptation of feature detectors. arXiv preprint arXiv:1207.0580. · arXiv
- Johannes Hoffart, Mohamed Amir Yosef, Ilaria Bordino, Hagen Fürstenau, Manfred Pinkal, Marc Spaniol, Bilyana Taneva, Stefan Thater, and Gerhard Weikum. 2011. Robust disambiguation of named entities in text. In Proceedings of the Conference on Empirical Meth- ods in Natural Language Processing, pages 782-792. Association for Computational Linguistics.
- Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidi- rectional LSTM-CRF models for sequence tagging. CoRR, abs/1508.01991. · arXiv
- Yoon Kim, Yacine Jernite, David Sontag, and Alexan- der M. Rush. 2015. Character-aware neural language models. CoRR, abs/1508.06615. · arXiv
- John Lafferty, Andrew McCallum, and Fernando CN Pereira. 2001. Conditional random fields: Probabilis- tic models for segmenting and labeling sequence data. In Proc. ICML.
- Dekang Lin and Xiaoyun Wu. 2009. Phrase clustering for discriminative learning. In Proceedings of the Joint dings for named entity resolution. arXiv preprint arXiv:1404.5367. · arXiv
- Yanjun Qi, Ronan Collobert, Pavel Kuksa, Koray Kavukcuoglu, and Jason Weston. 2009. Combining labeled and unlabeled data with word-class distribu- tion learning. In Proceedings of the 18th ACM con- ference on Information and knowledge management, pages 1737-1740. ACM.
- Lev Ratinov and Dan Roth. 2009. Design challenges and misconceptions in named entity recognition. In Proceedings of the Thirteenth Conference on Compu- tational Natural Language Learning, pages 147-155. Association for Computational Linguistics.
- Cicero Nogueira dos Santos and Victor Guimarães. 2015. Boosting named entity recognition with neural charac- ter embeddings. arXiv preprint arXiv:1505.05008. · arXiv
- Erik F. Tjong Kim Sang and Fien De Meulder. 2003. In- troduction to the conll-2003 shared task: Language- independent named entity recognition. In Proc. CoNLL.
- Erik F. Tjong Kim Sang. 2002. Introduction to the conll- 2002 shared task: Language-independent named entity recognition. In Proc. CoNLL.
- Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: A simple and general method for semi-supervised learning. In Proc. ACL.
- Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classi- fication. In Advances in Neural Information Process- ing Systems, pages 649-657.
- Jie Zhou and Wei Xu. 2015. End-to-end learning of se- mantic role labeling using recurrent neural networks. In Proceedings of the Annual Meeting of the Associa- tion for Computational Linguistics.
Cited by
Sorted by citations
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · see citing works
Recent Trends in Deep Learning Based Natural Language Processing [Review Article]
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2840738 · Cited by 2,921 · see citing works
A Survey on Knowledge Graphs: Representation, Acquisition, and Applications
Ji, S., Pan, S., Cambria, E., Marttinen, P., & Yu, P. S. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3070843 · Cited by 2,947 · see citing works
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
Ma, X., & Hovy, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1101 · Cited by 2,567 · see citing works
Attention-based LSTM for Aspect-level Sentiment Classification
Wang, Y., Huang, M., zhu, X., & Zhao, L. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1058 · Cited by 2,450 · see citing works
Deep Learning With Edge Computing: A Review
Chen, J., & Ran, X. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2921977 · Cited by 1,540 · see citing works
How Multilingual is Multilingual BERT?
Pires, T., Schlinger, E., & Garrette, D. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1493 · Cited by 1,351 · see citing works
A large language model for electronic health records
Yang, X., Chen, A., PourNejatian, N., Shin, H. C., Smith, K. E., Parisien, C., Compas, C., Martin, C., Costa, A. B., Flores, M. G., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00742-2 · Cited by 847 · see citing works
Peng, Y., Yan, S., & Lu, Z. (2019) · Proceedings of the 18th BioNLP Workshop and Shared Task · doi:10.18653/v1/w19-5006 · Cited by 899 · see citing works
Marshall, I. J., & Wallace, B. C. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1074-9 · Cited by 638 · see citing works
“…are based on neural networks, and specifically recurrent neural networks, or RNNs. Long short-term memory networks (LSTMs) [19] combined with CRFs (LSTM-CRFs) [19 – 21] have in particular shown compelling performance on such tasks generally, for extraction of data from RCTs specifically [22, 23].”
BackgroundA Unified MRC Framework for Named Entity Recognition
Li, X., Feng, J., Meng, Y., Han, Q., Wu, F., & Li, J. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.519 · Cited by 697 · see citing works
Identifying Customer Needs from User-Generated Content
Timoshenko, A., & Hauser, J. R. (2019) · Marketing Science · doi:10.1287/mksc.2018.1123 · Cited by 513 · see citing works
Template-Based Named Entity Recognition Using BART
Cui, L., Wu, Y., Liu, J., Yang, S., & Zhang, Y. (2021) · arXiv · doi:10.48550/arxiv.2106.01760 · Cited by 342 · see citing works
“Third, compared with traditional meth-ods (pre-trained model with a softmax/CRF), our method can be applied to arbitrary new categories of named entities without changing the output layer, and therefore allows continual learning (Lin et al., 2020).”
BackgroundMethodInfluentialLearning How to Ask: Querying LMs with Mixtures of Soft Prompts
Qin, G., & Eisner, J. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.410 · Cited by 342 · see citing works
Latent Predictor Networks for Code Generation
Ling, W., Blunsom, P., Grefenstette, E., Hermann, K. M., Kočiský, T., Wang, F., & Senior, A. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1057 · Cited by 307 · see citing works
A Unified Generative Framework for Various NER Subtasks
Yan, H., Gui, T., Dai, J., Guo, Q., Zhang, Z., & Qiu, X. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.451 · Cited by 299 · see citing works
“…labelling formulation, which will assign a tag to each token in the sentence, has been widely used in the flat NER field (McCal-lum and Li, 2003; Collobert et al., 2011; Huang et al., 2015; Chiu and Nichols, 2016; Lample et al., 2016; Strakov´a et al., 2019; Yan et al., 2019; Li et al., 2020a).”
MethodStructured prediction models for RNN based sequence labeling in clinical text
Jagannatha, A., & yu, H. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1082 · Cited by 215 · see citing works
Harvesting Paragraph-level Question-Answer Pairs from Wikipedia
Du, X., & Cardie, C. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1177 · Cited by 168 · see citing works
Extracting comprehensive clinical information for breast cancer using deep learning methods
Zhang, X., Zhang, Y., Zhang, Q., Ren, Y., Qiu, T., Ma, J., & Sun, Q. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.103985 · Cited by 160 · see citing works
Assessing The Factual Accuracy of Generated Text
Goodrich, B., Rao, V., Liu, P. J., & Saleh, M. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330955 · Cited by 162 · see citing works
Recent advances in biomedical literature mining
Zhao, S., Su, C., Lu, Z., & Wang, F. (2021) · Briefings in Bioinformatics · doi:10.1093/bib/bbaa057 · Cited by 134 · see citing works
Cyberthreat Detection from Twitter using Deep Neural Networks
Dionisio, N., Alves, F., Ferreira, P. M., & Bessani, A. (2019) · 2019 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852475 · Cited by 126 · see citing works
SpanNER: Named Entity Re-/Recognition as Span Prediction
Fu, J., Huang, X., & Liu, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.558 · Cited by 112 · see citing works
Biomedical Named Entity Recognition with Multilingual BERT
Hakala, K., & Pyysalo, S. (2019) · Proceedings of The 5th Workshop on BioNLP Open Shared Tasks · doi:10.18653/v1/d19-5709 · Cited by 96 · see citing works
Anonymisation Models for Text Data: State of the art, Challenges and Future Directions
Lison, P., Pilán, I., Sanchez, D., Batet, M., & Øvrelid, L. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.323 · Cited by 80 · see citing works
Improving reference prioritisation with PICO recognition.
Brockmeier, A. J., Ju, M., Przybyła, P., & Ananiadou, S. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0992-8 · Cited by 62 · see citing works
Campillos-Llanos, L., Valverde-Mateos, A., Capllonch-Carrión, A., & Moreno-Sandoval, A. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01395-z · Cited by 65 · see citing works
APE: Argument Pair Extraction from Peer Review and Rebuttal via Multi-task Learning
Cheng, L., Bing, L., Yu, Q., Lu, W., & Si, L. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.569 · Cited by 58 · see citing works
Extracting Symptoms and their Status from Clinical Conversations
Du, N., Chen, K., Kannan, A., Tran, L., Chen, Y., & Shafran, I. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1087 · Cited by 66 · see citing works
A pre-training and self-training approach for biomedical named entity recognition
Gao, S., Kotevska, O., Sorokine, A., & Christian, J. B. (2021) · PLOS ONE · doi:10.1371/journal.pone.0246310 · Cited by 46 · see citing works
Automated Emerging Cyber Threat Identification and Profiling Based on Natural Language Processing
Marinho, R., & Holanda, R. (2023) · IEEE Access · doi:10.1109/access.2023.3260020 · Cited by 50 · see citing works
Pretraining to Recognize PICO Elements from Randomized Controlled Trial Literature.
Tian, K., Shirui, Z., & Chunhua, W. (2019) · Studies in Health Technology and Informatics · doi:10.3233/shti190209 · Cited by 45 · see citing works
Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling
Liu, L., Ren, X., Shang, J., Gu, X., Peng, J., & Han, J. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1153 · Cited by 45 · see citing works
Yu, Z., Yang, X., Dang, C., Wu, S., Adekkanattu, P., Pathak, J., George, T. J., Hogan, W. R., Guo, Y., Bian, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.04949 · Cited by 38 · see citing works
A General Framework for Information Extraction using Dynamic Span Graphs
Luan, Y., Wadden, D., He, L., Shah, A., Ostendorf, M., & Hajishirzi, H. (2019) · arXiv · doi:10.48550/arxiv.1904.03296 · Cited by 34 · see citing works
“Entity recognition has commonly been cast as a sequence labeling problem, and has benefited substantially from the use of neural architectures (Collobert et al., 2011; Lample et al., 2016; Ma and Hovy, 2016; Luan et al., 2017b, 2018b).”
BackgroundMethodInfluentialYang, X., Zhang, H., He, X., Bian, J., & Wu, Y. (2020) · JMIR Medical Informatics · doi:10.2196/22982 · Cited by 29 · see citing works
An Enhanced Neural Word Embedding Model for Transfer Learning
Kowsher, M., Sobuj, M. S. I., Shahriar, M. F., Prottasha, N. J., Arefin, M. S., Dhar, P. K., & Koshiba, T. (2022) · Applied Sciences · doi:10.3390/app12062848 · Cited by 27 · see citing works
Stylianou, N., Razis, G., Goulis, D. G., & Vlahavas, I. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101949 · Cited by 22 · see citing works
Tiftikci, M., Özgür, A., He, Y., & Hur, J. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3195-5 · Cited by 23 · see citing works
Tri-Train: Automatic Pre-Fine Tuning between Pre-Training and Fine-Tuning for SciNER
Zeng, Q., Yu, W., Yu, M., Jiang, T., Weninger, T., & Jiang, M. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.429 · Cited by 10 · see citing works
Azab, M., Dadian, S., Nastase, V., An, L., & Mihalcea, R. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1122 · Cited by 5 · see citing works
- Authors
- Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., & Dyer, C.
- Affiliations
- Universitat Pompeu Fabra (ES)
- ORCID
- Ballesteros, Miguel
- Venue
- Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Published
- 2016
- Pages
- 260-270
- OpenAlex
- W2296283641
- Cited by
- 4,401
- References
- 4
- Citations by year
- 2026: 148 · 2025: 211 · 2024: 334 · 2023: 479 · 2022: 495 · 2021: 652 · 2020: 692 · 2019: 667 · 2018: 443 · 2017: 209 · 2016: 71
- FWCI
- 249.66
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- Repositori digital de la UPF (Universitat Pompeu Fabra) (publishedVersion, cc-by-nc-sa)
- Funders
- European Commission (FP7 ICT, 610411, H2020, 645012)
- Defense Advanced Research Projects Agency (HR0011-15-C-0114)
- Advanced Research Projects Agency
- Horizon 2020 Framework Programme
- FP7 Information and Communication Technologies
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- named entity recognition, neural architecture, sequence labeling, recurrent neural networks, bidirectional LSTM, character-level features
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
