The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Named Entity Recognition with Bidirectional LSTM-CNNs
Open accessAbstract
Named entity recognition is a challenging task that has traditionally required large amounts of knowledge in the form of feature engineering and lexicons to achieve high performance. In this paper, we present a novel neural network architecture that automatically detects word- and character-level features using a hybrid bidirectional LSTM and CNN architecture, eliminating the need for most feature engineering. We also propose a novel method of encoding partial lexicon matches in neural networks and compare it to existing approaches. Extensive evaluation shows that, given only tokenized text and publicly available word embeddings, our system is competitive on the CoNLL-2003 dataset and surpasses the previously reported state of the art performance on the OntoNotes 5.0 dataset by 2.13 F1 points. By using two lexicons constructed from publicly-available sources, we establish new state of the art performance with an F1 score of 91.62 on CoNLL-2003 and 86.28 on OntoNotes, surpassing systems that employ heavy feature engineering, proprietary lexicons, and rich entity linking information.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 7×
“…quantities of unlabelled text – an approach made possible by recent advancements in unsupervised learning of word embeddings on massive amounts of data (Collobert and Weston, 2008; Mikolov et al., 2013) and neural network training algorithms permitting deep architectures (Rumelhart et al., 1986).”
MethodInfluentialThe particle swarm - explosion, stability, and convergence in a multidimensional complex space
Clerc, M., & Kennedy, J. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.985692 · Cited by 9,017
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · cited 7×
“Following the speech-recognition framework out-lined by Graves et al. (2013), we employed a stacked 1 bi-directional recurrent neural network with long short-term memory units to transform word features into named entity tag scores.”
BackgroundMethodInfluentialOn the Properties of Neural Machine Translation: Encoder–Decoder Approaches
Cho, K., van Merrienboer, B., Bahdanau, D., & Bengio, Y. (2014) · Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation · doi:10.3115/v1/w14-4012 · Cited by 7,099
ADADELTA: An Adaptive Learning Rate Method
Zeiler, M. D. (2012) · arXiv · doi:10.48550/arxiv.1212.5701 · Cited by 5,322
A unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
Natural Language Processing (almost) from Scratch
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 4,962
Learning to forget: Continual prediction with LSTM
Gers, F. A., Schmidhuber, J., & Cummins, F. (2000) · Neural Computation · doi:10.1162/089976600300015015 · Cited by 4,823
Cited by
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Neural Architectures for Named Entity Recognition
Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., & Dyer, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1030 · Cited by 4,401
A Survey on Knowledge Graphs: Representation, Acquisition, and Applications
Ji, S., Pan, S., Cambria, E., Marttinen, P., & Yu, P. S. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3070843 · Cited by 2,947
Recent Trends in Deep Learning Based Natural Language Processing [Review Article]
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2018) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2018.2840738 · Cited by 2,921
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
Ma, X., & Hovy, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1101 · Cited by 2,567
A Survey of the Usages of Deep Learning for Natural Language Processing
Otter, D. W., Medina, J. R., & Kalita, J. K. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2979670 · Cited by 1,836
A Survey on Deep Learning for Named Entity Recognition
Li, J., Sun, A., Han, J., & Li, C. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2981314 · Cited by 1,567
Recent Trends in Deep Learning Based Natural Language Processing
Young, T., Hazarika, D., Poria, S., & Cambria, E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.02709 · Cited by 1,192
Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling
Marcheggiani, D., & Titov, I. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1159 · Cited by 907
Chinese NER Using Lattice LSTM
Zhang, Y., & Yang, J. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1144 · Cited by 878
Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme
Zheng, S., Wang, F., Bao, H., Hao, Y., Zhou, P., & Xu, B. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1113 · Cited by 787
A Unified MRC Framework for Named Entity Recognition
Li, X., Feng, J., Meng, Y., Han, Q., Wu, F., & Li, J. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.519 · Cited by 697
Detecting Hate Speech on Twitter Using a Convolution-GRU Based Deep Neural Network
Zhang, Z., Robinson, D., & Tepper, J. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-93417-4_48 · Cited by 638
Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations
Kiperwasser, E., & Goldberg, Y. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00101 · Cited by 615
Harnessing Deep Neural Networks with Logic Rules
Hu, Z., Ma, X., Liu, Z., Hovy, E., & Xing, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1228 · Cited by 545
Identifying Customer Needs from User-Generated Content
Timoshenko, A., & Hauser, J. R. (2019) · Marketing Science · doi:10.1287/mksc.2018.1123 · Cited by 513
Fast and Accurate Entity Recognition with Iterated Dilated Convolutions
Strubell, E., Verga, P., Belanger, D., & McCallum, A. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1283 · Cited by 474
A Survey on Recent Advances in Named Entity Recognition from Deep Learning models
Yadav, V., & Bethard, S. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.11470 · Cited by 464
Named Entity Recognition as Dependency Parsing
Yu, J., Bohnet, B., & Poesio, M. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.577 · Cited by 458
A Joint Neural Model for Information Extraction with Global Features
Lin, Y., Ji, H., Huang, F., & Wu, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.713 · Cited by 383
Deep Active Learning for Named Entity Recognition
Shen, Y., Yun, H., Lipton, Z., Kronrod, Y., & Anandkumar, A. (2017) · Proceedings of the 2nd Workshop on Representation Learning for NLP · doi:10.18653/v1/w17-2630 · Cited by 370
Semi-Supervised Sequence Modeling with Cross-View Training
Clark, K., Luong, M.-T., Manning, C. D., & Le, Q. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1217 · Cited by 366
Results of the WNUT2017 Shared Task on Novel and Emerging Entity Recognition
Derczynski, L., Nichols, E., van Erp, M., & Limsopatham, N. (2017) · Proceedings of the 3rd Workshop on Noisy User-generated Text · doi:10.18653/v1/w17-4418 · Cited by 361
Self-citationTemplate-Based Named Entity Recognition Using BART
Cui, L., Wu, Y., Liu, J., Yang, S., & Zhang, Y. (2021) · arXiv · doi:10.48550/arxiv.2106.01760 · Cited by 342
“, 2019) is used to represent the input text, and softmax (Chiu and Nichols, 2016; Strubell et al., 2017; Cui and Zhang, 2019) or CRF (Lample et al.”
MethodInfluentialEmpower Sequence Labeling with Task-Aware Neural Language Model
Liu, L., Shang, J., Ren, X., Xu, F., Gui, H., Peng, J., & Han, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12006 · Cited by 333
Template-Based Named Entity Recognition Using BART
Cui, L., Wu, Y., Liu, J., Yang, S., & Zhang, Y. (2021) · Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 · doi:10.18653/v1/2021.findings-acl.161 · Cited by 314
A Unified Generative Framework for Various NER Subtasks
Yan, H., Gui, T., Dai, J., Guo, Q., Zhang, Z., & Qiu, X. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.451 · Cited by 299
“The sequence labelling formulation, which will assign a tag to each token in the sentence, has been widely used in the flat NER field (McCallum and Li, 2003; Collobert et al., 2011; Huang et al., 2015; Chiu and Nichols, 2016; Lample et al., 2016; Straková et al., 2019; Yan et al., 2019; Li et al., 2020a).”
BackgroundMethodInfluentialVisual Attention Model for Name Tagging in Multimodal Social Media
Lu, D., Neves, L., Carvalho, V., Zhang, N., & Ji, H. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1185 · Cited by 298
LLMs for knowledge graph construction and reasoning: recent capabilities and future opportunities
Zhu, Y., Wang, X., Chen, J., Qiao, S., Ou, Y., Yao, Y., Deng, S., Chen, H., & Zhang, N. (2024) · World Wide Web · doi:10.1007/s11280-024-01297-w · Cited by 294
“Typically, KG construction (Ye et al., 2022b) consists of several tasks, including Named Entity Recognition (NER) (Chiu and Nichols, 2016), Relation Extraction (RE) (Zeng et al., 2015; Chen et al., 2022), Event Extraction (EE) (Chen et al., 2015; Deng et al., 2020), and Entity Linking (EL) (Shen et…”
Simplify the Usage of Lexicon in Chinese NER
Ma, R., Peng, M., Zhang, Q., Wei, Z., & Huang, X. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.528 · Cited by 294
Character-Based LSTM-CRF with Radical-Level Features for Chinese Named Entity Recognition
Dong, C., Zhang, J., Zong, C., Hattori, M., & Di, H. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-50496-4_20 · Cited by 284
Yu, J., Jiang, J., Yang, L., & Xia, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.306 · Cited by 281
Cross-type biomedical named entity recognition with deep multi-task learning
Wang, X., Zhang, Y., Ren, X., Zhang, Y., Zitnik, M., Shang, J., Langlotz, C., & Han, J. (2019) · Bioinformatics · doi:10.1093/bioinformatics/bty869 · Cited by 273
Evaluation of Deep Learning Models for Multi-Step Ahead Time Series Prediction
Chandra, R., Goyal, S., & Gupta, R. (2021) · IEEE Access · doi:10.1109/access.2021.3085085 · Cited by 265
Leveraging Knowledge Bases in LSTMs for Improving Machine Reading
Yang, B., & Mitchell, T. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1132 · Cited by 261
MasakhaNER: Named Entity Recognition for African Languages
Adelani, D. I., Abbott, J., Neubig, G., D’souza, D., Kreutzer, J., Lignos, C., Palen-Michel, C., Buzaaba, H., Rijhwani, S., Ruder, S., et al. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00416 · Cited by 257
Structured prediction models for RNN based sequence labeling in clinical text
Jagannatha, A., & yu, H. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1082 · Cited by 215
Assessing The Factual Accuracy of Generated Text
Goodrich, B., Rao, V., Liu, P. J., & Saleh, M. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330955 · Cited by 162
“Entities can be identified with named-entity recognition (NER) systems like Chiu and Nichols [5], Finkel et al. [7], Lample et al. [9] that take in 𝑋 and produce the set { 𝑒 } .”
BackgroundArtificial intelligence for anti-money laundering: a review and extension
Han, J., Huang, Y., Liu, S., & Towey, K. (2020) · Digital Finance · doi:10.1007/s42521-020-00023-1 · Cited by 113
SpanNER: Named Entity Re-/Recognition as Span Prediction
Fu, J., Huang, X., & Liu, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.558 · Cited by 112
Anonymisation Models for Text Data: State of the art, Challenges and Future Directions
Lison, P., Pilán, I., Sanchez, D., Batet, M., & Øvrelid, L. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.323 · Cited by 80
Extracting Symptoms and their Status from Clinical Conversations
Du, N., Chen, K., Kannan, A., Tran, L., Chen, Y., & Shafran, I. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1087 · Cited by 66
MasakhaNER: Named entity recognition for African languages
(2021) · HAL · doi:10.1162/tacl · Cited by 61
APE: Argument Pair Extraction from Peer Review and Rebuttal via Multi-task Learning
Cheng, L., Bing, L., Yu, Q., Lu, W., & Si, L. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.569 · Cited by 58
Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling
Liu, L., Ren, X., Shang, J., Gu, X., Peng, J., & Han, J. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1153 · Cited by 45
Why pay more? A simple and efficient named entity recognition system for tweets
Suman, C., Reddy, S. M., Saha, S., & Bhattacharyya, P. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114101 · Cited by 27
Uncertainty-Aware Machine Support for Paper Reviewing on the Interspeech 2019 Submission Corpus
Stappen, L., Rizos, G., Hasan, M., Hain, T., & Schuller, B. W. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-2862 · Cited by 15
Azab, M., Dadian, S., Nastase, V., An, L., & Mihalcea, R. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1122 · Cited by 5
“Sequence labelling approaches have proved successful for such problems (Ramshaw and Marcus, 1995; Lample et al., 2016; Ma and Hovy, 2016; Chiu and Nichols, 2016).”
BackgroundEstablishing trust for using natural language for intent-based networking
Jacobs, A. S., Pfitscher, R. J., Ribeiro, R. H., Granville, L. Z., Ferreira, R. A., Willinger, W., & Rao, S. G. (2025) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2025.3574626 · Cited by 5
“L UMI ’ S NER architecture is described in detail in [14] and uses an enhanced version of the Long Short-Term Memory (LSTM) model ( i.e., the so-called Bi-LSTM model [39]).”
Joint extraction of entity and relation based on fine-tuning BERT for long biomedical literatures
Gao, T., Zhai, X., Yang, C., Lv, L., & Wang, H. (2024) · Bioinformatics Advances · doi:10.1093/bioadv/vbae194 · Cited by 2
Leveraging Foundation Models for Clinical Text Analysis
Raza, S., & Bashir, S. R. (2023) · 2023 IEEE 11th International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi57859.2023.00103 · Cited by 2
Metadata
- Authors
- Chiu, J. P. C., & Nichols, E.
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press - Journals
- Published
- 2016-12
- Volume
- 4
- Pages
- 357-370
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W2179519966
- Cited by
- 1,909
- References
- 33
- FWCI
- 167.36
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.