E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT
Open accessAbstract
We present a novel way of injecting factual knowledge about entities into the pretrained BERT model ( Devlin et al., 2019): We align Wikipedia2Vec entity vectors (Yamada et al., 2016) with BERT's native wordpiece vector space and use the aligned entity vectors as if they were wordpiece vectors.The resulting entity-enhanced version of BERT (called E-BERT) is similar in spirit to ERNIE (Zhang et al., 2019) and KnowBert (Peters et al., 2019), but it requires no expensive further pretraining of the BERT encoder.We evaluate E-BERT on unsupervised question answering (QA), supervised relation classification (RC) and entity linking (EL).On all three tasks, E-BERT outperforms BERT and other baselines.We also show quantitatively that the original BERT model is overly reliant on the surface form of entity names (e.g., guessing that someone with an Italian-sounding name speaks Italian), and that E-BERT mitigates this problem.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Large language models have the ability to retrieve factual knowledge as explicit knowledge bases, presenting an opportunity to construct more comprehensive knowledge graphs within recommender systems.
“The ability of Large Language Models to retrieve factual knowledge as explicit knowledge bases [53-56, 56-61] has been stirred discussed, which presents an opportunity to construct more comprehensive knowledge graphs within recommender systems.”
From When large language models meet personalization: perspectives of challenges and opportunities (2024) · cites this paper as [58]
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278 · see citing works
Self-citationXLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · see citing works
Exploiting Similarities among Languages for Machine TranslationPreprint
Mikolov, T., Le, Q. V., & Sutskever, I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1309.4168 · Cited by 1,502 · see citing works
ERNIE: Enhanced Language Representation with Informative Entities
Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., & Liu, Q. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1139 · Cited by 1,482 · see citing works
Fixing Weight Decay Regularization in Adam
Loshchilov, I., & Hutter, F. (2018) · Cited by 1,297 · see citing works
How Can We Know What Language Models Know?
Jiang, Z., Xu, F. F., Araki, J., & Neubig, G. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00324 · Cited by 1,002 · see citing works
Robust Disambiguation of Named Entities in Text
Hoffart, J., Yosef, M. A., Bordino, I., Fürstenau, H., Pinkal, M., Spaniol, M., Taneva, B., Thater, S., & Weikum, G. (2011) · Cited by 922 · see citing works
ERNIE 2.0: A Continual Pre-Training Framework for Language Understanding
Sun, Y., Wang, S., Li, Y., Feng, S., Tian, H., Wu, H., & Wang, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6428 · Cited by 766 · see citing works
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation
Wang, X., Gao, T., Zhu, Z., Zhang, Z., Liu, Z., Li, J., & Tang, J. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00360 · Cited by 676 · see citing works
Knowledge Enhanced Contextual Word Representations
Peters, M. E., Neumann, M., Logan, R., Schwartz, R., Joshi, V., Singh, S., & Smith, N. A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1005 · Cited by 661 · see citing works
Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text
Sun, H., Dhingra, B., Zaheer, M., Mazaitis, K., Salakhutdinov, R., & Cohen, W. W. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1455 · Cited by 510 · see citing works
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters
Wang, R., Tang, D., Duan, N., Wei, Z., Huang, X., Ji, J., Cao, G., Jiang, D., & Zhou, M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.121 · Cited by 503 · see citing works
Mask-Predict: Parallel Decoding of Conditional Masked Language Models
Ghazvininejad, M., Levy, O., Liu, Y., & Zettlemoyer, L. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1633 · Cited by 464 · see citing works
Offline bilingual word vectors, orthogonal transformations and the inverted softmaxPreprint
Smith, S. L., Turban, D. H. P., Hamblin, S., & Hammerla, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.03859 · Cited by 398 · see citing works
Language Models as Knowledge Bases?Preprint
Petroni, F., Rocktäschel, T., Lewis, P., Bakhtin, A., Wu, Y., Miller, A., & Riedel, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01066 · Cited by 364 · see citing works
Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation
Yamada, I., Shindo, H., Takeda, H., & Takefuji, Y. (2016) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1025 · Cited by 337 · see citing works
Commonsense Knowledge Mining from Pretrained Models
Davison, J., Feldman, J., & Rush, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1109 · Cited by 264 · see citing works
StructBERT: Incorporating Language Structures into Pre-training for Deep Language UnderstandingPreprint
Wang, W., Bi, B., Yan, M., Wu, C., Bao, Z., Xia, J., Peng, L., & Si, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.04577 · Cited by 243 · see citing works
End-to-End Neural Entity Linking
Kolitsas, N., Ganea, O.-E., & Hofmann, T. F. (2018) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k18-1050 · Cited by 242 · see citing works
Show Your Work: Improved Reporting of Experimental Results
Dodge, J., Gururangan, S., Card, D., Schwartz, R., & Smith, N. A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1224 · Cited by 233 · see citing works
(2018) · doi:10.63317/2yu5nu76n7bu · Cited by 189 · see citing works
Question Answering on Knowledge Bases and Text using Universal Schema and Memory Networks
Das, R., Zaheer, M., Reddy, S., & McCallum, A. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-2057 · Cited by 156 · see citing works
Investigating Entity Knowledge in BERT with Simple Neural End-To-End Entity Linking
Broscheit, S. (2019) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k19-1063 · Cited by 114 · see citing works
BERTRAM: Improved Word Embeddings Have Big Impact on Contextualized Model Performance
Schick, T., & Schütze, H. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.368 · Cited by 43 · see citing works
Self-citationImproving Pre-Trained Multilingual Model with Vocabulary Expansion
Wang, H., Yu, D., Sun, K., Chen, J., & Yu, D. (2019) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k19-1030 · Cited by 34 · see citing works
Rare Words: A Major Problem for Contextualized Embeddings and How to Fix it by Attentive Mimicking
Schick, T., & Schütze, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6403 · Cited by 12 · see citing works
Self-citationImproving Pre-Trained Multilingual Models with Vocabulary ExpansionPreprint
Wang, H., Yu, D., Sun, K., Chen, J., & Yu, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.12440 · Cited by 4 · see citing works
Contextualized End-to-End Neural Entity LinkingPreprint
Chen, H., Žukov-Gregorič, A., Li, X. D., & Wadhwa, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.03834 · Cited by 4 · see citing works
Flagged first, then by citations
1 of the 33 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
A primer in BERTology: What we know about how BERT works
Rogers, A., Kovaleva, O., & Rumshisky, A. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00349 · Cited by 1,552 · see citing works
Pre-trained models for natural language processing: A survey
Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., & Huang, X. (2020) · Science China Technological Sciences · doi:10.1007/s11431-020-1647-3 · Cited by 1,591 · see citing works
How Can We Know What Language Models Know?
Jiang, Z., Xu, F. F., Araki, J., & Neubig, G. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00324 · Cited by 1,002 · see citing works
Pre-trained models: Past, present and future
Han, X., Zhang, Z., Ding, N., Gu, Y., Liu, X., Huo, Y., Qiu, J., Yao, Y., Zhang, A., Zhang, L., et al. (2021) · AI Open · doi:10.1016/j.aiopen.2021.08.002 · Cited by 994 · see citing works
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters
Wang, R., Tang, D., Duan, N., Wei, Z., Huang, X., Ji, J., Cao, G., Jiang, D., & Zhou, M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.121 · Cited by 503 · see citing works
KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation
Wang, X., Gao, T., Zhu, Z., Zhang, Z., Liu, Z., Li, J., & Tang, J. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00360 · Cited by 676 · see citing works
When large language models meet personalization: perspectives of challenges and opportunities
Chen, J., Liu, Z., Huang, X., Wu, C., Liu, Q., Jiang, G., Pu, Y., Lei, Y., Chen, X., Wang, X., et al. (2024) · World Wide Web · doi:10.1007/s11280-024-01276-1 · Cited by 367 · see citing works
Mallen, A., Asai, A., Zhong, V., Das, R., Khashabi, D., & Hajishirzi, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.546 · Cited by 340 · see citing works
Yang, L., Chen, H., Li, Z., Ding, X., & Wu, X. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3360454 · Cited by 238 · see citing works
True Few-Shot Learning with Language Models
Perez, E., Kiela, D., & Cho, K. (2021) · arXiv · doi:10.48550/arxiv.2105.11447 · Cited by 226 · see citing works
Measuring and improving consistency in pretrained language models
Elazar, Y., Kassner, N., Ravfogel, S., Ravichander, A., Hovy, E., Schütze, H., & Goldberg, Y. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00410 · Cited by 220 · see citing works
Self-citationDomain Adaptation via Prompt Learning
Ge, C., Huang, R., Xie, M., Lai, Z., Song, S., Li, S., & Huang, G. M. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3327962 · Cited by 223 · see citing works
Jiang, Z., Araki, J., Ding, H., & Neubig, G. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00407 · Cited by 177 · see citing works
AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language ProcessingPreprint
Kalyan, K. S., Rajasekharan, A., & Sangeetha, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.05542 · Cited by 240 · see citing works
Information-Theoretic Probing with Minimum Description Length
Voita, E., & Titov, I. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.14 · Cited by 197 · see citing works
Neural entity linking: A survey of models based on deep learning
Sevgili, Ö., Shelmanov, A., Arkhipov, M., Panchenko, A., & Biemann, C. (2022) · Semantic Web · doi:10.3233/sw-222986 · Cited by 171 · see citing works
How Context Affects Language Models' Factual Predictions
Petroni, F., Lewis, P., Piktus, A., Rocktäschel, T., Wu, Y., Miller, A. H., & Riedel, S. (2020) · arXiv · doi:10.48550/arxiv.2005.04611 · Cited by 154 · see citing works
Calibration of Transformer-Based Models for Identifying Stress and Depression in Social Media
Ilias, L., Mouzakitis, S., & Askounis, D. (2023) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2023.3283009 · Cited by 97 · see citing works
X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models
Jiang, Z., Anastasopoulos, A., Araki, J., Ding, H., & Neubig, G. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.479 · Cited by 88 · see citing works
Heinzerling, B., & Inui, K. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.153 · Cited by 78 · see citing works
Meta-learning via Language Model In-context Tuning
Chen, Y., Zhong, R., Zha, S., Karypis, G., & He, H. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.53 · Cited by 84 · see citing works
Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases
Cao, B., Lin, H., Han, X., Sun, L., Yan, L., Liao, M., Xue, T., & Xu, J. (2021) · doi:10.18653/v1/2021.acl-long.146 · Cited by 90 · see citing works
Can Language Models be Biomedical Knowledge Bases?
Sung, M., Lee, J., Yi, S. S., Jeon, M., Kim, S., & Kang, J. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.388 · Cited by 76 · see citing works
Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning
Tamborrino, A., Pellicanò, N., Pannier, B., Voitot, P., & Naudin, L. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.357 · Cited by 60 · see citing works
Towards Continual Knowledge Learning of Language Models
Jang, J., Ye, S., Yang, S., Shin, J., Han, J., Kim, G., Choi, S. J., & Seo, M. (2021) · arXiv · doi:10.48550/arxiv.2110.03215 · Cited by 49 · see citing works
“We also remove instances where the answer overlapped with the subject following Poerner et al. (2019) since the answers for these instances can be inferred from the cloze statement itself.”
BackgroundBootleg: Chasing the Tail with Self-Supervised Named Entity DisambiguationPreprint
Orr, L., Leszczynski, M., Arora, S., Wu, S., Guha, N., Xiao, L., & Ré, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.10363 · Cited by 48 · see citing works
Jiang, Z., Araki, J., Ding, H., & Neubig, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.00955 · Cited by 39 · see citing works
Yamada, I., Asai, A., Sakuma, J., Shindo, H., Takeda, H., Takefuji, Y., & Matsumoto, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.06280 · Cited by 52 · see citing works
Combining BERT with Static Word Embeddings for Categorizing Social Media
Alghanmi, I., Espinosa-Anke, L., & Schockaert, S. (2020) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/2020.wnut-1.5 · Cited by 34 · see citing works
Empowering Language Models with Knowledge Graph Reasoning for Open-Domain Question Answering
Hu, Z., Yi‐chong, X., Yu, W., Wang, S., Yang, Z., Zhu, C., Chang, K.-W., & Sun, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.650 · Cited by 42 · see citing works
Relational World Knowledge Representation in Contextual Language Models: A Review
Safavi, T., & Koutra, D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.81 · Cited by 37 · see citing works
EcomGPT: Instruction-Tuning Large Language Models with Chain-of-Task Tasks for E-commerce
Li, Y., Ma, S., Wang, X., Huang, S., Jiang, C., Zheng, H.-T., Xie, P., Huang, F., & Jiang, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29820 · Cited by 38 · see citing works
How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis
Li, S., Li, X., Shang, L., Dong, Z., Sun, C., Liu, B., Ji, Z., Jiang, X., & Liu, Q. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.136 · Cited by 34 · see citing works
Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic KnowledgePreprint
Verga, P., Sun, H., Soares, L. B., & Cohen, W. W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.00849 · Cited by 36 · see citing works
Verlinden, S., Zaporojets, K., Deleu, J., Demeester, T., & Develder, C. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.171 · Cited by 30 · see citing works
A survey on knowledge-enhanced multimodal learning
Lymperaiou, M., & Stamou, G. B. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10825-z · Cited by 35 · see citing works
Entity-aware Transformers for Entity Search
Gerritse, E. J., Hasibi, F., & de Vries, A. P. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531971 · Cited by 28 · see citing works
García-Barragán, Á., Sakor, A., Vidal, M.-E., Menasalvas, E., Gonzalez, J. C. S., Provencio, M., & Robles, V. (2024) · Medical & Biological Engineering & Computing · doi:10.1007/s11517-024-03227-4 · Cited by 25 · see citing works
Li, R., Jiang, Z., Wang, L., Lu, X., Zhao, M., & Chen, D. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.106936 · Cited by 26 · see citing works
Knowledge Enhanced Pretrained Language Models: A Compreshensive SurveyPreprint
Wei, X., Wang, S., Zhang, D., Bhatia, P., & Arnold, A. O. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08455 · Cited by 38 · see citing works
Suresh, V., & Ong, D. C. (2021) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii52823.2021.9597390 · Cited by 25 · see citing works
Fusing external knowledge resources for natural language understanding techniques: A survey
Wang, Y., Wang, W., Chen, Q., Huang, K., Nguyen, A., De, S., & Hussain, A. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.11.025 · Cited by 22 · see citing works
Graph Machine Learning in the Era of Large Language Models (LLMs)
Wang, S., Huang, J., Chen, Z., Song, Y., Tang, W., Mao, H., Fan, W., Liu, H., Liu, X., Yin, D., et al. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3732786 · Cited by 39 · see citing works
Structure-inducing pre-training
McDermott, M. B. A., Yap, B., Szolovits, P., & Žitnik, M. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00647-z · Cited by 23 · see citing works
Meng, Z., Liu, F., Shareghi, E., Su, Y., Collins, C., & Collier, N. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.329 · Cited by 20 · see citing works
Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal View
Cao, B., Lin, H., Han, X., Liu, F., & Sun, L. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.398 · Cited by 22 · see citing works
Cross-Domain Aspect Extraction using Transformers Augmented with Knowledge Graphs
Howard, P., Ma, A., Lal, V., Simoes, A. P., Korat, D., Pereg, O., Wasserblat, M., & Singer, G. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557275 · Cited by 19 · see citing works
Chatterjee, S., & Dietz, L. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531944 · Cited by 18 · see citing works
The Life Cycle of Knowledge in Big Language Models: A Survey
Cao, B., Lin, H., Han, X., & Sun, L. (2024) · Machine Intelligence Research · doi:10.1007/s11633-023-1416-x · Cited by 18 · see citing works
The Life Cycle of Knowledge in Big Language Models: A Survey
Cao, B., Lin, H., Han, X., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2303.07616 · Cited by 18
García-Olano, D., Onoe, Y., & Ghosh, J. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3487553.3524648 · Cited by 17 · see citing works
Evaluating Language Models for Knowledge Base Completion
Veseli, B., Singhania, S., Razniewski, S., & Weikum, G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-33455-9_14 · Cited by 16 · see citing works
Al-Zoubi, A. M., Mora, A. M., & Faris, H. (2023) · IEEE Access · doi:10.1109/access.2023.3293641 · Cited by 20 · see citing works
Enhancing Multilingual Language Model with Massive Multilingual Knowledge Triples
Liu, L., Li, X., He, R., Bing, L., Joty, S., & Si, L. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.462 · Cited by 18 · see citing works
Language Models As or For Knowledge Bases
Razniewski, S., Yates, A., Kassner, N., & Weikum, G. (2021) · arXiv · doi:10.48550/arxiv.2110.04888 · Cited by 16 · see citing works
Understanding Transformer Memorization Recall Through Idioms
Haviv, A., Cohen, I., Gidron, J., Schuster, R., Goldberg, Y., & Geva, M. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.19 · Cited by 19 · see citing works
DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learning
Bai, S., Zhang, J., Guo, S., Li, S. C., Guo, J., Hou, J., Han, T., & Lu, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02576 · Cited by 14 · see citing works
Detection, Disambiguation, Re-ranking: Autoregressive Entity Linking as a Multi-Task Problem
Mrini, K., Nie, S., Gu, J., Wang, S., Sanjabi, M., & Firooz, H. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.156 · Cited by 15 · see citing works
Impact of Co-occurrence on Factual Knowledge of Large Language Models
Kang, C., & Choi, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.518 · Cited by 18 · see citing works
Garg, A., Gupta, S., Vats, S., Handa, P., & Goel, N. (2024) · Expert Systems · doi:10.1111/exsy.13697 · Cited by 13 · see citing works
Conversational Entity Linking: Problem Definition and Datasets
Joko, H., Hasibi, F., Balog, K., & de Vries, A. P. (2021) · NORA - Norwegian Open Research Archives · Cited by 13 · see citing works
Pre-training Language Models with Deterministic Factual Knowledge
Li, S., Li, X., Shang, L., Sun, C., Liu, B., Ji, Z., Jiang, X., & Liu, Q. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.764 · Cited by 14 · see citing works
Exploiting global context and external knowledge for distantly supervised relation extraction
Gao, J., Wan, H., & Lin, Y. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110195 · Cited by 13 · see citing works
Khowaja, S. A., Nkenyereye, L., Khuwaja, P., Hamadi, H. A., & Dev, K. (2024) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2024.3461776 · Cited by 13 · see citing works
Static Embeddings as Efficient Knowledge Bases?
Dufter, P., Kassner, N., & Schütze, H. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.186 · Cited by 13 · see citing works
Self-citationConversational Entity Linking: Problem Definition and Datasets
Joko, H., Hasibi, F., Balog, K., & de Vries, A. P. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3463258 · Cited by 12 · see citing works
KgPLM: Knowledge-guided Language Model Pre-training via Generative and Discriminative LearningPreprint
He, B., Jiang, X., Xiao, J., & Liu, Q. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.03551 · Cited by 13 · see citing works
P-Adapters: Robustly Extracting Factual Information from Language Models with Diverse PromptsPreprint
Newman, B., Choubey, P. K., & Rajani, N. F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.07280 · Cited by 12 · see citing works
Islam, S. M., & Bhattacharya, S. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3511941 · Cited by 12 · see citing works
Plug-and-Play Knowledge Injection for Pre-trained Language Models
Zhang, Z., Zeng, Z., Lin, Y., Wang, H., Ye, D., Xiao, C., Han, X., Liu, Z., Li, P., Sun, M., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.594 · Cited by 17 · see citing works
Heinzerling, B., & Inui, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.09036 · Cited by 11 · see citing works
App review driven collaborative bug finding
Tang, X., Tian, H., Kong, P., Ezzini, S., Liu, K., Xia, X., Klein, J., & Bissyandé, T. F. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10489-x · Cited by 10 · see citing works
A Simple but Effective Pluggable Entity Lookup Table for Pre-trained Language Models
Ye, D., Lin, Y., Li, P., Sun, M., & Liu, Z. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.57 · Cited by 11 · see citing works
SpEL: Structured Prediction for Entity Linking
Shavarani, H. S., & Sarkar, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.686 · Cited by 11 · see citing works
Metadata Shaping: A Simple Approach for Knowledge-Enhanced Language Models
Arora, S., Wu, S., Liu, E., & Ré, C. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.137 · Cited by 10 · see citing works
Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases
Cao, B., Lin, H., Han, X., Sun, L., Yan, L., Liao, M., Xue, T., & Xu, J. (2021) · arXiv · doi:10.48550/arxiv.2106.09231 · Cited by 10 · see citing works
LM-CORE: Language Models with Contextually Relevant External Knowledge
Kaur, J. N., Bhatia, S., Aggarwal, M., Bansal, R., & Krishnamurthy, B. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.57 · Cited by 9 · see citing works
KG-prompt: Interpretable knowledge graph prompt for pre-trained language models
Chen, L., Liu, J., Duan, Y., & Wang, R. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113118 · Cited by 9 · see citing works
Exploring Pre-Trained Language Models to Build Knowledge Graph for Metal-Organic Frameworks (MOFs)
An, Y., Greenberg, J., Hu, X. T., Kalinowski, A., Fang, X., Zhao, X., McClellan, S., Uribe‐Romo, F. J., Langlois, K., Furst, J. D., et al. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020568 · Cited by 9 · see citing works
Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models
Youssef, P., Koraş, O. A., Li, M., Schlötterer, J., & Seifert, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1043 · Cited by 13 · see citing works
Linking Entities to Unseen Knowledge Bases with Arbitrary Schemas
Vyas, Y., & Ballesteros, M. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.65 · Cited by 11 · see citing works
Hengle, A., Kshirsagar, A., Desai, S., & Marathe, M. V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.05683 · Cited by 9 · see citing works
How to Query Language Models?Preprint
Adolphs, L., Dhuliawala, S., & Hofmann, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.01928 · Cited by 10 · see citing works
Transparency Helps Reveal When Language Models Learn Meaning
Wu, Z., Merrill, W., Peng, H. M., Beltagy, I., & Smith, N. A. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00565 · Cited by 8 · see citing works
M-Tuning: Prompt Tuning With Mitigated Label Bias in Open-Set Scenarios
Liao, N., Zhang, X., Cao, M., & Yan, J. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3524181 · Cited by 8 · see citing works
ECIR 23 Tutorial: Neuro-Symbolic Approaches for Information Retrieval
Dietz, L., Bast, H., Chatterjee, S., Dalton, J., Meij, E., & de Vries, A. P. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28241-6_33 · Cited by 7 · see citing works
A Multilingual Bag-of-Entities Model for Zero-Shot Cross-Lingual Text Classification
Nishikawa, S., Yamada, I., Tsuruoka, Y., & Echizen, I. (2022) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2022.conll-1.1 · Cited by 7 · see citing works
LMTurk: Few-Shot Learners as Crowdsourcing Workers in a Language-Model-as-a-Service Framework
Zhao, M., Mi, F., Wang, Y., Li, M., Jiang, X., Liu, Q., & Schuetze, H. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.51 · Cited by 8 · see citing works
Commonsense Evidence Generation and Injection in Reading Comprehension
Liu, Y., Yang, T., You, Z., Fan, W., & Yu, P. S. (2020) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2020.sigdial-1.9 · Cited by 11 · see citing works
Does Knowledge Help General NLU? An Empirical StudyPreprint
Xu, R., Fang, Y., Zhu, C., & Zeng, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.00563 · Cited by 8 · see citing works
Xu, N., Wang, F., Li, B., Dong, M., & Chen, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.592 · Cited by 6 · see citing works
An Empirical Study on Few-shot Knowledge Probing for Pretrained Language ModelsPreprint
He, T., Cho, K., & Glass, J. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.02772 · Cited by 7 · see citing works
Evaluating the Knowledge Base Completion Potential of GPT
Veseli, B., Razniewski, S., Kalo, J.-C., & Weikum, G. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.426 · Cited by 5 · see citing works
Towards Efficient Methods in Medical Question Answering using Knowledge Graph Embeddings
Sengupta, S., Heaton, C., Cui, S., Sarkar, S., & Mitra, P. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10821824 · Cited by 7 · see citing works
Improving Zero-Shot Entity Retrieval through Effective Dense Representations
Partalidou, E., Christou, D., & Tsoumakas, G. (2022) · Hellenic Conference on Artificial Intelligence · doi:10.1145/3549737.3549771 · Cited by 6 · see citing works
Selective UMLS knowledge infusion for biomedical question answering
Park, H., Son, J., Min, J., & Choi, J. (2023) · Scientific Reports · doi:10.1038/s41598-023-41423-8 · Cited by 4 · see citing works
Leveraging Emotion-Specific features to improve Transformer performance for Emotion Classification
Desai, S., Kshirsagar, A., Sidnerlikar, A., Khodake, N., & Marathe, M. V. (2022) · Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA) · doi:10.18653/v1/2022.wassa-1.24 · Cited by 4 · see citing works
Enriching Contextualized Representations with Biomedical Ontologies: Extending KnowBert to UMLS
Piat, G., Semmar, N., Allauzen, A., Essafi, H., & Tourille, J. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-10464-0_52 · Cited by 4 · see citing works
Knowledge Based Multilingual Language ModelPreprint
Liu, L., Li, X., He, R., Bing, L., Joty, S., & Si, L. (2021) · arXiv (Cornell University) · Cited by 5 · see citing works
- Authors
- Poerner, N., Waltinger, U., & Schütze, H.
- Affiliations
- Siemens (Germany) (DE)
- Ludwig-Maximilians-Universität München (DE)
- Venue
- Findings of the Association for Computational Linguistics: EMNLP 2020
- Published
- 2020
- Pages
- 803-818
- OpenAlex
- W3021191241
- Cited by
- 146
- References
- 33
- Citations by year
- 2026: 5 · 2025: 11 · 2024: 21 · 2023: 24 · 2022: 34 · 2021: 34 · 2020: 15 · 2019: 1 · 2018: 1
- FWCI
- 10.21
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Siemens
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- entity embeddings, BERT, knowledge injection, relation classification, entity linking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
