CoChat

Named Entity Recognition and Relation Extraction: State-of-the-Art

Nasar, Zara, Jaffry, Syed Waqar, Malik, Muhammad Kamran · 2022 · ACM Computing Surveys

The DOI matches its Crossref record; 75 of the 78 references it cites were found in a registry or an index.

Abstract

With the advent of Web 2.0, there exist many online platforms that result in massive textual-data production. With ever-increasing textual data at hand, it is of immense importance to extract information nuggets from this data. One approach towards effective harnessing of this unstructured textual data could be its transformation into structured text. Hence, this study aims to present an overview of approaches that can be applied to extract key insights from textual data in a structured way. For this, Named Entity Recognition and Relation Extraction are being majorly addressed in this review study. The former deals with identification of named entities, and the latter deals with problem of extracting relation between set of entities. This study covers early approaches as well as the developments made up till now using machine learning models. Survey findings conclude that deep-learning-based hybrid and joint models are currently governing the state-of-the-art. It is also observed that annotated benchmark datasets for various textual-data generators such as Twitter and other social forums are not available. This scarcity of dataset has resulted into relatively less progress in these domains. Additionally, the majority of the state-of-the-art techniques are offline and computationally expensive. Last, with increasing focus on deep-learning frameworks, there is need to understand and explain the under-going processes in deep architectures.

Overview

Integrity signals

Registry

Crossref · 10.1145/3445965

Cites

75 of 75 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 75 references

Research works printed but not linked: 83

  1. Muslea I., 1999, Proceedings of the AAAI Workshop on Machine Learning for Information Extraction.
  2. Simoes G., Proceedings of the INForum.
  3. National Institute of Standards and Technology (NIST)., 2017, Proceedings of the Text Analytic Conference.
  4. Augenstein I., 2017, Apr.
  5. Merchant R., 1996, Proceedings of a Workshop on held at Vienna
  6. Asian Federation of Natural Language Processing., 2008, IJCNLP-08 Workshop on NER for South and South East Asian Languages.
  7. Peng N., Proceedings of the Conference on Empirical Methods in Natural Language Processing. 548--554
  8. Wang W., Proceedings of the International Conference on Asian Language Processing (IALP’15)
  9. Kanya N., Proceedings of the IET Chennai 3rd International on Sustainable Energy and Intelligent Systems (SEISCON’12)
  10. Palshikar G. K., 2013, Techniques for named entity recognition. Bioinforma. Concepts Methodol. Tools Appl. 400
  11. Rizzolo N., Proceedings of the International Conference on Semantic Computing (ICSC’07)
  12. Klein D., Proceedings of the 7th Conference on Natural Language Learning at HLT-NAACL, 4
  13. Liao W., 1829, Proceedings of the NAACL HLT Workshop on Semi-Supervised Learning for Natural Language Processing. 58--65
  14. 10.5555/1090483.1644538 · doi:10.5555/1090483.1644538
  15. Passos A., 2014, Apr.
  16. Peters M., Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1756--1765, 17
  17. Rondeau M. A., Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU’15)
  18. Strubell E., 2017, Feb.
  19. Ageishi R., Proceedings of the 1st International Conference on the Applications of Digital Information and Web Technologies (ICADIWT’08)
  20. McCallum A., Proceedings of the 7th Conference on Natural Language Learning at HLT-NAACL, 4
  21. Yu X., Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing Association for Computational Linguistics. 3073--3077
  22. Khashabi D., 2018, Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC’18)
  23. Strassel S., Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC’16)
  24. Cucerzan S., Proceedings of the 6th Conference on Natural Language Learning, 20
  25. Leonandya R. A., Proceedings of the 3rd International Symposium on Computational and Business Intelligence (ISCBI’15)
  26. Straková J., Proceedings of the International Conference on Text, Speech, and Dialogue. 173--181
  27. Liu L., 2018, Proceedings of the AAAI Conference on Artificial Intelligence.
  28. Doan X.-D., Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation.
  29. Lample G., Proceedings of NAACL-HLT, Association for Computational Linguistics. 260--270, 16
  30. 10.5555/2390426.2390435 · doi:10.5555/2390426.2390435
  31. Täckström O., 2096, Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies.
  32. Qu L., Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics. 899--905, 16
  33. Chen L., 2018, Proceedings of the 2nd Workshop on Natural Language for Artificial Intelligence (NL4AI’18)
  34. P. Cao Y. Chen K. Liu J. Zhao and S. Liu. 2018. Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism. 182--192. Retrieved from https://aclweb.org/anthology/papers/D/D18/D18-1017/. P. Cao Y. Chen K. Liu J. Zhao and S. Liu. 2018. Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism. 182--192. Retrieved from https://aclweb.org/anthology/papers/D/D18/D18-1017/.
  35. Rahimi A., Proceedings of the 57th Conference of the Association for Computational Linguistics. 151--164
  36. Bharadwaj A., Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1462--1472
  37. Xie J., Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 369--379, 18
  38. Z. Nasar S. W. Jaffry and M. K. Malik. 2018. Information extraction from scientific articles: A survey. Scientometrics. DOI:10.1007/s11192-018-2921-5 · DOI
  39. Abdelmagid M., 2014, Survey on information extraction from chemical compound literatures: Techniques and challenges, J. Theor. Appl. Inf. Technol., 67, 284
  40. Bonadiman D., Proceedings of the 2nd Italian Conference on Computational Linguistics (CLiC It’15)
  41. J. Li et al. 2016. BioCreative V CDR task corpus: A resource for chemical disease relation extraction. Datab.- J. Biol. Datab. Curat. May (2016). DOI:10.1093/database/baw068 · DOI · Paper page
  42. Dong X., Proceedings of the New York Scientific Data Summit (NYSDS’16)
  43. Finkel J. R., Proceedings of the Conference on Empirical Methods in Natural Language Processing, 1
  44. Lu W., Proceedings of the Conference on Empirical Methods in Natural Language Processing. 857--867
  45. Katiyar A., Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1
  46. Ju M., Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1
  47. Mai K., 2018, Proceedings of the 27th International Conference on Computational Linguistics. 711--722
  48. F. Li M. Zhang G. Fu and D. Ji. 2017. A neural joint model for entity and relation extraction from biomedical text. BMC Bioinf. 18 (2017). DOI:10.1186/s12859-017-1609-9 · DOI · Paper page
  49. N. Konstantinova. 2014. Review of relation extraction methods: What is new out there? In Analysis of Images Social Networks and Texts. 15--28. DOI:10.1007/978-3-319-12580-0_2 · DOI
  50. Asghar N., 2016, Automatic extraction of causal relations from natural language texts: A comprehensive survey. ArXiv Prepr. ArXiv160507895
  51. Brin S., 1998, Proceedings of the International Workshop on the World Wide Web and Databases. 172--183
  52. Agichtein E., Proceedings of the 5th ACM Conference on Digital Libraries. 85--94
  53. McDonald R., Proceedings of the 43rd Meeting on Association for Computational Linguistics. 491--498
  54. Kumar S., 2017, May
  55. A. Smirnova and P. Cudré-Mauroux. 2018. Relation extraction using distant supervision: A survey. ACM Comput. Surv. 51 5 (2018) 106:1--106:35. DOI:10.1145/3241741 · DOI · Paper page
  56. Nédellec C., 2005, Proceedings of the 4th Learning Language in Logic Workshop (LLL’05)
  57. 2013, Proceedings of the 2013th International Conference on NLP 8 DBpedia (NLP-DBPEDIA'13), 1064
  58. Leroy G., Proceedings of the Pacific Symposium on Biocomputing.
  59. GuoDong Z., Proceedings of the 43rd Meeting on Association for Computational Linguistics. 427--434
  60. Chan Y. S., 2002, Proceedings of the 49th Meeting of the Association for Computational Linguistics: Human Language Technologies, 1
  61. Zelenko D., Proceedings of the ACL Conference on Empirical Methods in Natural Language Processing, 10
  62. Carlson A., Proceedings of the 3rd ACM International Conference on Web Search and Data Mining. 101--110
  63. De S., Proceedings of the Conference on Empirical Methods in Natural Language Processing.
  64. Riedel S., Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (HLT-NAACL’13)
  65. Mintz M., Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, 2
  66. Riedel S., Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. 148--163
  67. Takamatsu S., 2012, Reducing wrong labels in distant supervision for relation extraction. In Proceedings of the 50th Meeting of the Association for Computational Linguistics, Long Papers, 1, 721
  68. Sekine S., 2005, Proceedings of the International Workshop on Paraphrasing (IWP’05)
  69. Downey D., Proceedings of the Meeting of the Association for Computational Linguistics.
  70. Miwa M., 1869, Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14). 1858
  71. Mejer A., Proceedings of the Conference on Empirical Methods in Natural Language Processing. 971--981
  72. Crammer K., Proceedings of the Conference on Advances in Neural Information Processing Systems. 414--422
  73. Zeng D., 2014, Proceedings of the International Conference on Computational Linguistics (COLING’14)
  74. S. Takase N. Okazaki and K. Inui. 2016. Modeling semantic compositionality of relational patterns. Eng. Appl. Artif. Intell. 50 C (2016) 256--264. DOI:10.1016/j.engappai.2016.01.027 · DOI
  75. Zhang X., 2019, Proceedings of the AAAI Conference on Artificial Intelligence. 33
  76. J. Qu D. Ouyang W. Hua Y. Ye and X. Li. 2018. Distant supervision for neural relation extraction integrated with word attention and property features. Neural Netw. 100 C (2018) 59--69. DOI:10.1016/j.neunet.2018.01.006 · DOI
  77. Li Y., 2076, Proceedings of the 2nd International Conference on Computer Science and Application Engineering. 119:1--119:7. DOI:10
  78. Li Q., Proceedings of the Meeting of the Association for Computational Linguistics. 402--412
  79. Di S., Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery 8 Data Mining. 1348--1357
  80. Ling X., Proceedings of the AAAI Conference on Artificial Intelligence. 94--100
  81. Liu T., Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2195--2204, 18
  82. Hasegawa T., Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics. 415--422
  83. Yao L., 1870, Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1013--1023

Cited by

Items 201–289 of 289

Metadata

Authors
Nasar, Z., Jaffry, S. W., & Malik, M. K.
Affiliations
University of the Punjab (PK)
Venue
ACM Computing Surveys
Published
2021-02-11
Volume
54
Issue
1
Pages
1-39
ISSN
  • 0360-0300 (ISSN-L)
  • 1557-7341
OpenAlex
W3131198033
Cited by
245
References
78
Citations by year
2026: 35 · 2025: 69 · 2024: 63 · 2023: 47 · 2022: 28 · 2021: 3
FWCI
12.46
Type
journal article
Language
en
Open access
closed
Locations
ACM Computing Surveys (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
Keywords
named entity recognition, relation extraction, information extraction, unstructured text, social media text, Twitter, deep learning, hybrid model, joint modeling, benchmark dataset, explainability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.