Knowledge Enhanced Contextual Word Representations
Open accessThe DOI matches its Crossref record; 71 of the 72 references it cites were found in a registry or an index.
Abstract
Matthew E. Peters, Mark Neumann, Robert Logan, Roy Schwartz, Vidur Joshi, Sameer Singh, Noah A. Smith. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“XLM-R achieves state-of-the-arts results on multiple cross-lingual benchmarks, including XNLI, MLQA, and NER.”
From Pre-trained models for natural language processing: A survey (2020) · cites this paper as [62]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 4×
“We compute M m scores using the prior and dot product between the entity-span vectors and entity embeddings, with a two-layer MLP (100 hidden dimensions).”
BackgroundMethodInfluentialExploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 4×
“, 2018), and BERT (Devlin et al., 2019) have significantly improved the state of the art for a wide range of NLP tasks.”
BackgroundMethodInfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 4×
“, 2018), and BERT (Devlin et al., 2019) have significantly improved the state of the art for a wide range of NLP tasks.”
BackgroundMethodInfluentialGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 1×
“Pretrained word representations Initial work learning word vectors focused on static word embeddings using multi-task learning objectives (Collobert and Weston, 2008) or corpus level cooccurence statistics (Mikolov et al., 2013a; Pennington et al., 2014).”
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 3×
“Pretrained word representations Initial work learning word vectors focused on static word embeddings using multi-task learning objectives (Collobert and Weston, 2008) or corpus level co-occurence statistics (Mikolov et al., 2013a; Pen-nington et al., 2014).”
BackgroundDistributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 3×
“The entity embeddings use a skipgram like objective (Mikolov et al., 2013b) to learn 300-dimensional embeddings of Wikipedia”
BackgroundMethodWordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · cited 1×
“We experiment with integrating both WordNet (Miller, 1995) and Wikipedia, thus explicitly adding word sense knowledge and facts about named entities (including those unseen at training time).”
MethodDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · cited 6×
“Recently the field has shifted toward learning context-sensitive embeddings (Dai and Le, 2015; Peters et al., 2018; Devlin et al., 2019).”
BackgroundMethodInfluentialSelf-citation(2013) · Cited by 5,726
Research works printed but not linked: 5
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Cor- rado, and Jeff Dean. 2013b. Distributed representa- tions of words and phrases and their compositional- ity. In NeurIPS.
- Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word rep- resentations. In NAACL-HLT.
- Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In ACL.
- Bishan Yang and Tom Michael Mitchell. 2017. Lever- aging knowledge bases in LSTMs for improving ma- chine reading. In ACL.
- Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019. ERNIE: En- hanced language representation with informative en- tities. In ACL.
Cited by
Items 601–700 of 705
Transformer : des réseaux de neurones pour le traitement automatique des langues
Yvon, F. (2022) · Technologies logicielles Architectures des systèmes · doi:10.51257/a-v1-h3735
Multi-stage Knowledge Enhancement for Ultra-fine-grained Entity Typing
Yang, Y., & Li, N. (2022) · International Symposium on Autonomous Systems (ISAS) · doi:10.1109/isas55863.2022.9757340
“ERNIE [15] and Know-BERT [16] work based on similar idea to enhance CWRs using static entity embeddings separately learned from a knowledge base.”
BackgroundA Study on Entity Linking Across Domains”:" Which Data is Best for Fine-Tuning?
Soliman, H., Adel, H., Gad-Elrab, M. H., Milchevski, D., & Strötgen, J. (2022) · Workshop on Representation Learning for NLP (RepL4NLP) · doi:10.18653/v1/2022.repl4nlp-1.19
“In today’s research, the usage of pre-trained language models, such as BERT (Devlin et al., 2019), is particularly popular (Peters et al., 2019; Logeswaran et al., 2019; Humeau et al., 2020; Wu et al., 2020).”
BackgroundLatent graph learning with dual-channel attention for relation extraction
Tang, G., Li, P., He, Y., Chen, Y., Zhong, Y., & Gan, F. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.109471
Ruan, Z., Hou, X., & Jiang, L. (2022) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2022.semeval-1.179
“Know-BERT (Peters et al., 2019) used the knowledge embedding method and Zhang et al. make some improvements.”
MethodKnowledge-Guided Word Embedding Fine-Tuning ModelPreprint
Wei Kun, K. O. N. G., Racharak, T., Cao, Y., Peng, C., Ma, Q., & Nguyen, L.-M. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4178861
The Multi-Hot Representation-Based Language Model to Maintain Morpheme Units
Lee, J., Shin, J.-C., & Ock, C. (2022) · Applied Sciences · doi:10.3390/app122010612
Towards Language-driven Scientific AIPreprint
Gómez-Pérez, J. M. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.15327
“Finally, more research is required that goes beyond current methods (Peters et al., 2019; Wang et al., 2021) to validate the acquired scientific knowledge, inject such knowledge into the model, and update the model’s beliefs on demand and possibly in real-time.”
BackgroundImage Captioning with External Knowledge
Nikiforova, S. (2023) · Utrecht University Repository (Utrecht University) · doi:10.33540/1709
Short Analysis of the Impact of COVID-19 Ontologies
Groppe, S., Tiwari, S. M., Khorashadizadeh, H., Groppe, J., Groth, T., Benamara, F., & Sahri, S. (2023) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-7126-6_17
Neural Approaches to Entity-Centric Information Extraction
Zaporojets, K. (2023) · Ghent University Academic Bibliography (Ghent University) · doi:10.48550/arxiv.2304.07625
Towards robust natural language and image processing In low-resource scenarios
Liu, L. (2023) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/166293
FREDA: Flexible Relation Extraction Data Annotation
Strobl, M., Trabelsi, A., & Zai͏̈ane, O. R. (2023) · ACM Symposium on Applied Computing · doi:10.1145/3555776.3578592
“[13], a knowledge-enhanced version of BERT through the integration of WordNet [12] and a subset of Wikipedia.”
BackgroundLovón-Melgarejo, J., Moreno, J. G., Besançon, R., Ferret, O., & Tamine, L. (2023) · HAL
Enrichissement des modèles de langue pré-entraînés par la distillation mutuelle des connaissances
Sourty, R., Moreno, J. G., Servant, F.-P., & Tamine, L. (2023) · HAL
Self-adaptive Prompt-tuning for Event Extraction in Ancient Chinese Literature
Zhang, J., Wei, Y., Zhu, Y., & Wu, B. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191495
“KnowBERT [19] introduces structured data into the pre-training process through entity recog-”
MethodImproving Relation Extraction by Entity-Level Contrastive Learning
Lu, T., Wang, S., Huang, Q., Guo, W., Chang, S., & Liu, G. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191841
“Reference [1] improves the performance of relation extraction by fusing external knowledge from the knowledge graph (KG).”
BackgroundMethodInfluentialExploring Automatically Perturbed Natural Language Explanations in Relation Extraction
Cui, W., & Chen, X. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.214
Using Two Losses and Two Datasets Simultaneously to Improve TempoWiC Accuracy
Pirhadi, M. J., Mirzaei, M., & Eetemadi, S. (2022) · doi:10.18653/v1/2022.evonlp-1.3
Integration of Heterogeneous Knowledge Sources for Biomedical Text Processing
Bagherzadeh, P., & Bergler, S. (2022) · International Workshop on Health Text Mining and Information Analysis (LOUHI) · doi:10.18653/v1/2022.louhi-1.25
“Thus, a majority of the current efforts focus on leveraging a single task relevant KS using hand-tailored architectures (Goodwin and Demner-Fushman, 2020; Peters et al., 2019; Bagherzadeh et al., 2018).”
BackgroundPRINCE: Prefix-Masked Decoding for Knowledge Enhanced Sequence-to-Sequence Pre-Training
Xu, S., Li, H., Yuan, P., Wu, Y., & He, X. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.171
“On the other hand, recent researches (Zhang et al., 2019; Peters et al., 2019; Xiong et al., 2020; Wang et al., 2019, 2020; Ke et al., 2020; Tian et al., 2020; Xu et al., 2021) demonstrate that enhancing PLMs with real-world knowledge is crucial for knowledge-driven downstream tasks, including…”
BackgroundCorpus-Based Task-Specific Relation Discovery
Ramanan, K. (2023) · doi:10.18653/v1/2023.matching-1.5
“Knowledge bases and semantic graphs see wide application in tasks such as question-answering (Saxena et al., 2020; Das et al., 2017), recommendation (Zhang et al., 2016), and natural language inference (Peters et al., 2019).”
BackgroundXie, Z., Ouyang, K., Xie, X., Zheng, H.-T., Liu, W., Huang, D., & Wu, B. (2023) · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · doi:10.18293/seke2023-200
Jose, J., & Soundarabai, P. B. (2023) · Algorithms for intelligent systems · doi:10.1007/978-981-99-4577-1_14
A Relational Classification Network Integrating Multi-scale Semantic Features
Li, G., Tian, J., Zhou, M., Li, M., & Han, D. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44696-2_11
An Effective Knowledgeable Label-Aware Approach for Sentential Relation Extraction
Nie, B., & Shao, Y. (2023) · Applied Sciences · doi:10.3390/app132111929
“Recent works on knowledge graph information sentential RE mainly focus on injecting external knowledge into PLMs. Methods such as ERNIE [13] and KnowBERT [14] incorporate pretrained entity embeddings from knowledge graphs into the transformer.”
BackgroundDemocratized Hyper-automated Software Development
Reddy, S. (2023) · The enterprise engineering series · doi:10.1007/978-3-031-29053-4_5
Research on Relation Extraction Based on BERT with Multifaceted Semantics
Dong, M., & Zhu, X. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-8181-6_13
Mannion, A., Schwab, D., Goeuriot, L., & Chevalier, T. (2023) · HAL
Integrating Noisy Knowledge into Language Representations for E-Commerce Applications
Samel, K., Ma, J., Wang, Z., Zhao, T., & Essa, I. A. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386615
“These KG-integrated Transformers [5]–[8] can leverage well-curated knowledge linked to their text representations.”
BackgroundSST: Semantic and Structural Transformers for Hierarchy-aware Language Models in E-commerce
Samel, K., Zhang, H., Ma, J., Jiang, H., Ping, Q., Wang, S., Xu, Y., Zeng, B., & Chilimbi, T. M. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386079
Attending to Entity Class Attributes for Named Entity Recognition with Few-Shot Learning
Patel, R. N., Dutta, S., & Assem, H. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-47715-7_57
FpRE:Front-prompt based dependency enhanced Few-shot Relation Extraction
Tan, Y., Zhou, Y., & Cheng, X. (2023) · International Conference on Systems and Informatics (ICSAI) · doi:10.1109/icsai61474.2023.10423340
“KNOWBERT [16] proposes to incorporate knowledge into input features,ALBERT [17] proposes a lightweight variant of cross-layer languge model represent adopts to coporate attention mechanism with merge description of relations and entities in text.”
BackgroundA Survey of Pre-Trained Language Models IncorporatingKnowledge Graphs
Wang, X., Liu, J., Zhou, J., & Wang, J. (2023) · IEEE International Conference on Electrical, Automation and Computer Engineering (ICEACE) · doi:10.1109/iceace60673.2023.10442824
“KnowBERT [18] introduces WordNet [19] and Wikipedia [20] to learn syntax and general knowledge, and fine-tune any downstream task; BERT-MK [21] is an improvement of ERNIE, which extracts the adjacent nodes of specific entities from the knowledge graph as supplementary entity background information,…”
MethodKnowledge graph population from news streams
Fernàndez-Cañellas, D. (2023) · UPCommons institutional repository (Universitat Politècnica de Catalunya) · doi:10.5821/dissertation-2117-403957
Graph-Enriched Biomedical Language Models: A Research Proposal
Sakhovskiy, A., Panchenko, A., & Tutubalina, E. V. (2023) · doi:10.18653/v1/2023.ijcnlp-srw.11
“First, they imply a modification of an LM architecture (Peters et al., 2019; Zhang et al., 2022b; Yasunaga et al., 2022a).”
BackgroundWhat does KnowBert-UMLS forget?
Piat, G., Semmar, N., Tourille, J., Allauzen, A., & Essafi, H. (2023) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa59173.2023.10479333
“KnowBert [5] is a BERT-based language model which alleviates this issue by including the EL task in its training objective.”
Knowledge Base Grounded Pre-trained Language Models via Distillation
Sourty, R., Moreno, J. G., Servant, F.-P., & Tamine, L. (2024) · ACM Symposium on Applied Computing · doi:10.1145/3605098.3635888
“Peters et al. [20] introduce the knowledge attention and contextualization component to incorporate factual knowledge from Wordnet synsets and design an entity linking pre-training objective.”
BackgroundImage classification with symbolic hints using limited resources
Jørgensen, M. G., Tětková, L., & Hansen, L. K. (2024) · PLoS ONE · doi:10.1371/journal.pone.0301360
“There is a growing interest in including knowledge bases or metadata in the learning process for hybrid models combining neural networks with symbolic knowledge (e.g. [7, 8]).”
BackgroundKrNER: A Novel Named Entity Recognition Method Based on Knowledge Enhancement and Remote Supervision
Du, J., & Yin, H. (2023) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom60117.2023.00328
Zhou, Z., Zhang, Q., & Zhao, F. (2024) · Cognitive Computation · doi:10.1007/s12559-024-10338-5
“KnowBERT [28] improves upon BERT by integrating multi-1 ple real-world knowledge bases into its pre-training process, with the aim of enhancing its coverage of real-world knowledge.”
Advanced methods for knowledge injection in large language models
Kulin, N., & Muravyov, S. (2024) · Scientific and technical journal of information technologies mechanics and optics · doi:10.17586/2226-1494-2024-24-4-588-593
Graph Neural Network for Molecular Structure: Application in HIV Inhibitor Molecule PredictionPreprint
Nguyen, Q. (2024) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/c3mqy
Knowledge-enhanced Multi-Granularity Interaction Network for Political Perspective Detection
Chen, X., Liu, X., Xu, Y., Wu, B., & Zhu, Y. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650944
“Methods like ERNIE[28], KnowBert[29], K-BERT[30] have been proposed.”
MethodEnhancing Pre-Trained Language Models with Knowledge Representation Using Line Graphs
Ge, Z., Zhu, Y., & Pan, R. (2024) · International Conference on Artificial Intelligence and Computer Information Technology (AICIT) · doi:10.1109/aicit62434.2024.10730175
“…entity auto-encoder pre-training task to inject knowledge embeddings into PLMs. • CokeBERT [17]: It can dynamically select contextual knowledge based on input text to enhance the PLMs. • KnowBERT [14]: Knowledge attention and recontextualization are used to inject the knowledge representations.”
BackgroundMethodConFit: Contrastive Fine-Tuning of Text-to-Text Transformer for Relation Classification
Duan, J., Lu, F., & Liu, J. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-9434-8_2
Li, X., & Wu, M. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0840-9_15
Wang, Y., Zhao, Y., Jin, G., Zhang, Z., Yin, F., Cui, R., & Li, M. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0847-8_9
Sentence-graph-level knowledge injection with multi-task learning
Chen, L., Wang, R., Chen, S., Yuan, Y., Liu, J., Hu, Y., & Jiang, F. (2025) · World Wide Web · doi:10.1007/s11280-025-01329-z
“KnowBert [47] presents a general embedding method to insert entity knowledge into the language model based on BERT.”
Gao, Y., Wang, S., Ren, F., & Wang, X. (2025) · Journal of Computer and Communications · doi:10.4236/jcc.2025.132007
“KnowBERT introduces a Knowledge Attention Representation (KAR) layer between two layers of the original BERT model, leveraging attention mechanisms among spans, between spans and entities, and among entities to facilitate knowledge interaction [13].”
A Stochastic Quasi-Newton Method for Non-convex Optimization with Non-uniform Smoothness
Sun, Z., & Wei, E. (2024) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc56724.2024.10886208
“The work [38] lays out a theoretical basis to better understand the superior performances of clipping-based algorithms.”
Constraint-Awareness and Graph Reasoning for Temporal Question Answering
Sun, Z., Zhang, K., Zhang, X., & Liu, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890668
“(I) PLMs includes BERT, KnowBERT [44] .”
Entity Linking Method Based on Longformer and Comparative Learning
赵, 占. (2025) · Computer Science and Application · doi:10.12677/csa.2025.155129
“( ) ˆ log 1 i y − ( ) [ ] 1 2 , , , L h h h 其中 t h 为第 t 个位置的上下文融合表示,进一步送入分类器进行最终的候选实体排序与实体链接判断。 ( ) t h ( ) x ( ) ( ) x ( ) x ( ) attention [9] 82.4 KnowBert [15] 73.7 BERT + Entity [6] 79.3 GENRE [14] 83.7 AT-LOM 85.5”
InfluentialFSDI: A Holistic Framework for Sustainability-Driven Innovation
Ullah, W., & Khan, M. A. (2025) · International Conference on Innovation in Artificial Intelligence and Internet of Things (AIIT) · doi:10.1109/aiit63112.2025.11082799
“Another key focus is training models on smaller datasets while maintaining high accuracy [36].”
Investigations on Automatic Question Answering Based on Knowledge Fusion
Wu, M., Zheng, W., Zhai, J., Li, B., & Duan, J. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-94892-3_30
K-BOOST: A Cyber Security NER Model with Knowledge Augmentation via BERT
Nie, R., Zhang, H., Ni, Y., Lian, Y., Liu, S., & Peng, Y. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-94458-1_10
CEEF: Contextual-Entity Embedding Fusion for Named Entity Recognition
Du, M., & Ye, W. (2025) · IEEE International Conference on Pattern Recognition and Machine Learning (PRML) · doi:10.1109/prml66062.2025.11159786
“ERNIE[15] and KnowBERT[16] enhance the performance of entity-relationship extraction by learning static entity embeddings from ex-ternal knowledge sources, which are then used to augment contextualized word representations, yielding improved results.”
Pham, T. T. T., & Bui, C. T. (2025) · Research Square · doi:10.21203/rs.3.rs-7883052/v1
Knowledge-Enhanced Vietnamese Paraphrase Identification
Xuan, M. L., Hong, M. N., Xuan, L., Thanh, T. D., Tien, D. B., & Minh, Q. T. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-4960-3_8
RehabKD With LLM-KG-LDACX: Orchestrating AI Models for Rehabilitation Knowledge Discovery
Zhang, W., Xing, Z., & Chen, S. (2025) · IEEE Access · doi:10.1109/access.2025.3645851
On Differential Privacy for Language Models
Arnold, S. (2024) · doi:10.25593/open-fau-1022
Pitchclass2vec: Symbolic Music Structure Segmentation with Chord Embeddings
Lazzari, N., Poltronieri, A., & Presutti, V. (2022) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.7868166
Efficient Natural Language Processing for Language Models
Xu, C. (2024) · eScholarship (California Digital Library) · Cited by 22
Structural Knowledge-Enhanced Approach for Network Device Log Understanding
曹, 祥. (2025) · Software Engineering and Applications · doi:10.12677/sea.2025.145090
A Physics Proxy-based Learning Framework for Relation Extraction on Discrete Data
Oh, J., & Cha, J.-W. (2025) · Journal of Broadcast Engineering · doi:10.5909/jbe.2025.30.6.858
Exploration of the Impact from Early Injection of Symbolic Knowledge into a Language Model
Machicao, J. C. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-10721-3_68
Zhou, B., Xu, Q., Che, K., Zhang, L., Cai, H., & Xing, L. (2026) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.70551
“…Roy and Pan [5] ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- © 2026 John Wiley & Sons Ltd. Peters et al. [6].”
Mazaheri, S., Dadashzadeh, A., Alankesh, A. N., & Dividino, R. (2025) · IEEE International Conference on Collaborative Advances in Software and Computing (CASCON) · doi:10.1109/cascon66301.2025.00112 · Cited by 1
“Research in combining LLMs with KGs has evolved from early work [20]–[23] embedding KGs into neural networks to more sophisticated integration methods.”
Document-Level Relation Extraction from Macro-Micro Perspective
Wen, X., Huo, Y., Shi, J., Su, M., Zhao, J., Li, W., Zuo, J., & Lu, Y. (2025) · IEEE International Conference on Data Science in Cyberspace · doi:10.1109/dsc67331.2025.00017
“KNOWBERT [17] proposes to incorporate knowledge into input features.”
Foolad, S., & Kiani, K. (2026) · Multimedia Tools and Applications · doi:10.1007/s11042-026-21221-w
“A different line of work [ 10–13 , 32– 34 ] improved transformer-based models by inject - ing them with external knowledge.”
Structured prediction for compute efficient and high accuracy NLP
Shavarani, H. S. (n.d.) · Summit (Simon Fraser University)
ModSolAgent: Automated Finite Element Code Generation for Abaqus via LLM-Based Agent
Li, Z., Ge, H., Tang, G., & Liu, Y. (2026) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2026.3669495 · Cited by 1
“Peters et al. [24] enhanced prediction accuracy by improving contextual word representations through knowledge integration.”
Liu, X., Lin, W., Hu, M., Xu, C., & Li, H. (2026) · Research Square · doi:10.21203/rs.3.rs-9084876/v1
Wang, Z., Zhang, G., Shi, N., & Fu, J. (2026) · Synthesis lectures on human language technologies · doi:10.1007/978-3-032-06264-2_9
Contextualization of Web contents through semantic enrichment from linked open data
Kumar, A. (2021) · theses.fr (ABES) · doi:10.70675/a7e0f151z9401z4beazb567zb2c9ddc4a3e5
Word Meaning Representation in Neural Language Models : Lexical Polysemy and Semantic Relationships
Soler, A. G. (2021) · theses.fr (ABES) · doi:10.70675/99a10411zc05ez4135z8f31z13573e0fd4d1
Towards efficient, general and robust entity disambiguation systems
Chen, L. (2023) · theses.fr (ABES) · doi:10.70675/abee8a0cz824dz4e34z917azffda9fe5cd83
Towards automatic understanding of narrative audiovisual content
Reboud, A. (2022) · theses.fr (ABES) · doi:10.70675/7b1296fez9f91z4484z8448z8040d28ab833 · Cited by 1
Représentations de phrases interprétables avec autoencodeurs variationnels et attention
Felhi, G. (2023) · theses.fr (ABES) · doi:10.70675/07f34446zf977z4a3bzad50z4145354b3316
Business relation extraction from texts
Khaldi, H. (2022) · theses.fr (ABES) · doi:10.70675/ce231ce4z5d07z4ab7z9758za927ea7b5302
A Reward-Driven Controller for Text Generation with Black-Box Language Models
Morain, R., & Ventura, D. A. (2025) · International Conference on Machine Learning and Applications · doi:10.1109/icmla66185.2025.00134
“This approach may be well suited to tasks aiming to integrate external knowledge into a LLM [35], [36], [37], [38].”
Une Approche Collaborative pour la Classification des Sites Web
Koufi, M. Z. (2024) · theses.fr (ABES) · doi:10.70675/1c35c522z32c3z4c0aza5eaz331e61ba1cd3
Improving Representations for Language Modeling
Godey, N. (2024) · theses.fr (ABES) · doi:10.70675/b6a15d0ez15bez43e5za967z16a069e0a8bb
Piat, G. (2023) · theses.fr (ABES) · doi:10.70675/bb18b592z47d5z40f0z805dz97e6aa90c6e5 · Cited by 1
Riabi, A. (2025) · theses.fr (ABES) · doi:10.70675/5a649ef5za082z43d8z906azd8051be91d15 · Cited by 4
Gyan: An Explainable Neuro-Symbolic Language ModelPreprint
Srinivasan, V., Jatav, V., Chandrababu, A., & Sharma, G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.04759 · Cited by 1
ConceptFormer: Towards Graph-Native Grounding of Large Language Models via Latent Concept Injection
Barmettler, J., Bernstein, A., & Rossetto, L. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774905.3794653
A Label-Aware and Knowledge-Augmented Framework for Multi-Label Text ClassificationPreprint
王, 联., 黄, 华., 杨, 嘉., 陈, 左., & 林, 南. (2026) · Research Square · doi:10.21203/rs.3.rs-9862072/v1
Rahi, P. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6949575
Knowledge-Aware BERT Architecture for Robust Hate Speech Identification
Saipavithira, M., Nivetha, S., & Manju, A. (2026) · International Conference on Innovations in Emerging Technologies for Sustainable Development (ICIETSD) · doi:10.1109/icietsd68684.2026.11584690
MECI: Multi-Element Collaborative Interaction for Multimodal Entity Linking
Peng, J., Shan, Y., Zha, Y., & Wang, X. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809583
“Local-level models [3, 23, 35, 42] focus on utilizing the immediate context, such as surrounding words or sentences, to aid disambiguation.”
Relation-Indicative Feature-Enhanced R-BERT for Entity Relation Classification
Duan, S., Yin, M., Luo, X., & Dai, S. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2759-4_19
Fake News Detection using BERT with Linguistic and Structural Handcrafted Features
Suleeman, S. K., & Ahmed, M. A. (2026) · International Symposium on AI-Driven Engineering Systems (ISADES) · doi:10.1109/isades69945.2026.11608189
Data and Knowledge Dual-Driven Text Embedding Approaches: A Survey
Chen, X., 詹佳青, Lin, J., Liu, H., Zhang, Q., Chen, J., & Chen, H. (2026) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2026.3721883
“KnowBert [25] incorporates multiple KBs into large models.”
Wu, J., Zhao, Y., Liao, Z., & Liu, M. (2026) · Knowledge and Information Systems · doi:10.1007/s10115-026-02880-5 · Cited by 1
“Pioneering work, such as that by Peters et al. [83], introduced knowledge attention and re-contextualization mechanisms to fuse entity embeddings from knowledge bases like Wikipedia and WordNet with BERT’s contextual representations, establishing an early technical path for knowledge injection.”
Metadata
- Authors
- Peters, M. E., Neumann, M., Logan, R., Schwartz, R., Joshi, V., Singh, S., & Smith, N. A.
- Affiliations
- Allen Institute for Artificial Intelligence (US)
- University of California, Irvine (US)
- University of Washington (US)
- Corresponding author
- Singh, Sameer
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 43-54
- OpenAlex
- W2970986510
- Cited by
- 661
- References
- 72
- Citations by year
- 2026: 21 · 2025: 54 · 2024: 81 · 2023: 125 · 2022: 115 · 2021: 160 · 2020: 98 · 2019: 6
- FWCI
- 36.26
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Funders
- National Science Foundation (1730158, 1817183)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- contextualized word representations, word embeddings, knowledge graphs, word sense disambiguation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.