The Life Cycle of Knowledge in Big Language Models: A Survey

Attention Is All You Need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works

Long Short-Term Memory

Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works

Language Models are Few-Shot Learners

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · see citing works

Freebase: a collaboratively created graph database for structuring human knowledge

Bollacker, K., Evans, C., Paritosh, P., Sturge, T., & Taylor, J. (2008) · Proceedings of the 2008 ACM SIGMOD international conference on Management of data · doi:10.1145/1376616.1376746 · Cited by 4,852 · see citing works

Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models

Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505 · see citing works

Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing

Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361 · see citing works

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308 · see citing works

Training language models to follow instructions with human feedback

Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works

Knowledge engineering: Principles and methods

Rudi Studer, A., Benjamins, V. R., & Fensel, D. (1998) · Data & Knowledge Engineering · doi:10.1016/s0169-023x(97)00056-6 · Cited by 3,697 · see citing works

Wikidata

Vrandečić, D., & Krötzsch, M. (2014) · Communications of the ACM · doi:10.1145/2629489 · Cited by 3,367 · see citing works

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · see citing works

The Power of Scale for Parameter-Efficient Prompt Tuning

Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096 · see citing works

PaLM: Scaling Language Modeling with Pathways

Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · see citing works

Equality of Opportunity in Supervised Learning

Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · see citing works

Prefix-Tuning: Optimizing Continuous Prompts for Generation

Li, X. L., & Liang, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.353 · Cited by 2,473 · see citing works

Language Models as Knowledge Bases?

Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1250 · Cited by 1,834 · see citing works

Semantics and complexity of SPARQL

Pérez, J., Arenas, M., & Gutiérrez, C. (2006) · TODS · doi:10.1145/1567274.1567278 · Cited by 1,678 · see citing works

Pre-trained models for natural language processing: A survey

Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., & Huang, X. (2020) · Science China Technological Sciences · doi:10.1007/s11431-020-1647-3 · Cited by 1,591 · see citing works

A primer in BERTology: What we know about how BERT works

Rogers, A., Kovaleva, O., & Rumshisky, A. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00349 · Cited by 1,552 · see citing works

What Does BERT Look at? An Analysis of BERT’s Attention

Clark, K., Khandelwal, U., Levy, O., & Manning, C. D. (2019) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w19-4828 · Cited by 1,514 · see citing works

ERNIE: Enhanced Language Representation with Informative Entities

Zhang, Z., Han, X., Liu, Z., Jiang, X., Sun, M., & Liu, Q. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1139 · Cited by 1,482 · see citing works

Knowledge engineering and management: The commonKADS methodology

(2000) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(00)90199-8 · Cited by 1,478 · see citing works

Scaling Instruction-Finetuned Language Models

Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441 · see citing works

Making Pre-trained Language Models Better Few-shot Learners

Gao, T., Fisch, A., & Chen, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.295 · Cited by 1,380 · see citing works

Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference

Schick, T., & Schütze, H. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.20 · Cited by 1,305 · see citing works

AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts

Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., & Singh, S. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.346 · Cited by 1,279 · see citing works

GPT understands, too

Liu, X., Zheng, Y., Du, Z., Ding, M., Qian, Y., Yang, Z., & Tang, J. (2023) · AI Open · doi:10.1016/j.aiopen.2023.08.012 · Cited by 1,124 · see citing works

Finetuned Language Models Are Zero-Shot Learners

Wei, J., Bosma, M., Zhao, V. Y., Guu, K., Yu, A. W., Lester, B., Du, N., Dai, A. M., & Le, Q. V. (2021) · arXiv · doi:10.48550/arxiv.2109.01652 · Cited by 1,101 · see citing works

How Can We Know What Language Models Know?

Jiang, Z., Xu, F. F., Araki, J., & Neubig, G. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00324 · Cited by 1,002 · see citing works

Counterfactual Fairness

Kusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06856 · Cited by 995 · see citing works

ERNIE: Enhanced Representation through Knowledge IntegrationPreprint

Sun, Y., Wang, S., Li, Y., Feng, S., Chen, X., Zhang, H., Tian, X., Zhu, D., Tian, H., & Wu, H. R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.09223 · Cited by 983 · see citing works

Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm

Reynolds, L., & McDonell, K. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3451760 · Cited by 952 · see citing works

COMET: Commonsense Transformers for Automatic Knowledge Graph Construction

Bosselut, A., Rashkin, H., Sap, M., Malaviya, C., Celikyilmaz, A., & Choi, Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1470 · Cited by 904 · see citing works

MASS: Masked Sequence to Sequence Pre-training for Language GenerationPreprint

Song, K., Tan, X., Qin, T., Lu, J., & Liu, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.02450 · Cited by 834 · see citing works

K-BERT: Enabling Language Representation with Knowledge Graph

Liu, W., Zhou, P., Zhao, Z., Wang, Z., Ju, Q., Deng, H., & Wang, P. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i03.5681 · Cited by 797 · see citing works

Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?

Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.759 · Cited by 755 · see citing works

Matching the Blanks: Distributional Similarity for Relation Learning

Baldini Soares, L., FitzGerald, N., Ling, J., & Kwiatkowski, T. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1279 · Cited by 731 · see citing works

KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language Representation

Wang, X., Gao, T., Zhu, Z., Zhang, Z., Liu, Z., Li, J., & Tang, J. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00360 · Cited by 676 · see citing works

Knowledge Enhanced Contextual Word Representations

Peters, M. E., Neumann, M., Logan, R., Schwartz, R., Joshi, V., Singh, S., & Smith, N. A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1005 · Cited by 661 · see citing works

Linguistic Knowledge and Transferability of Contextual Representations

Liu, N. F., Gardner, M., Belinkov, Y., Peters, M. E., & Smith, N. A. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1112 · Cited by 651 · see citing works

LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention

Yamada, I., Asai, A., Shindo, H., Takeda, H., & Matsumoto, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.523 · Cited by 647 · see citing works

How Much Knowledge Can You Pack into the Parameters of a Language Model?

Roberts, A., Raffel, C., & Shazeer, N. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.437 · Cited by 638 · see citing works

Multitask Prompted Training Enables Zero-Shot Task Generalization

Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.08207 · Cited by 612 · see citing works

Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity

Lu, Y., Bartolo, M., Moore, A., Riedel, S., & Stenetorp, P. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.556 · Cited by 610 · see citing works

Untitled work

(2019) · doi:10.18653/v1/n19-1419 · Cited by 601 · see citing works

REALM: Retrieval-Augmented Language Model Pre-Training

Guu, K., Lee, K., Tung, Z., Pasupat, P., & Chang, M.-W. (2020) · arXiv · doi:10.48550/arxiv.2002.08909 · Cited by 574 · see citing works

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.04615 · Cited by 569 · see citing works

What Makes Good In-Context Examples for GPT-3?

Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., & Chen, W. (2022) · Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures · doi:10.18653/v1/2022.deelio-1.10 · Cited by 563 · see citing works

Calibrate Before Use: Improving Few-Shot Performance of Language Models

Zhao, T. Z., Wallace, E., Feng, S., Klein, D., & Singh, S. (2021) · arXiv · doi:10.48550/arxiv.2102.09690 · Cited by 513 · see citing works

What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models

Ettinger, A. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00298 · Cited by 511 · see citing works

Perspectives on Cognitive Neuroscience

Churchland, P. S., & B, T. J. S. A. (1988) · Science · doi:10.1126/science.3055294 · Cited by 505 · see citing works

K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters

Wang, R., Tang, D., Duan, N., Wei, Z., Huang, X., Ji, J., Cao, G., Jiang, D., & Zhou, M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.121 · Cited by 503 · see citing works

PTR: Prompt Tuning with Rules for Text Classification

Han, X., Zhao, W., Ding, N., Liu, Z., & Sun, M. (2022) · AI Open · doi:10.1016/j.aiopen.2022.11.003 · Cited by 503 · see citing works

What do you learn from context? Probing for sentence structure in contextualized word representationsPreprint

Tenney, I., Xia, P., Chen, B., Wang, A., Poliak, A., McCoy, R. T., Kim, N., Durme, B. V., Bowman, S. R., Das, D., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.06316 · Cited by 480 · see citing works

BabelNet: Building a Very Large Multilingual Semantic Network

Roberto Navigli, A., & Ponzetto, S. P. (2010) · MADOC (University of Mannheim) · Cited by 466 · see citing works

Opt: Open pre-trained transformer language models

Zhang, S. (2025) · TIB Data Manager · doi:10.57702/dn0pmcbs · Cited by 405 · see citing works

Assessing BERT's Syntactic Abilities

Goldberg, Y. (2019) · arXiv · doi:10.48550/arxiv.1901.05287 · Cited by 385 · see citing works

Least-to-Most Prompting Enables Complex Reasoning in Large Language Models

Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.10625 · Cited by 380 · see citing works

Learning To Retrieve Prompts for In-Context Learning

Rubin, O., Herzig, J., & Berant, J. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.191 · Cited by 374 · see citing works

Avoiding Discrimination through Causal Reasoning

Kilbertus, N., Rojas-Carulla, M., Parascandolo, G., Hardt, M., Janzing, D., & Schölkopf, B. (2017) · arXiv · doi:10.48550/arxiv.1706.02744 · Cited by 358 · see citing works

Transformer Feed-Forward Layers Are Key-Value Memories

Geva, M., Schuster, R., Berant, J., & Levy, O. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.446 · Cited by 353 · see citing works

Emergent linguistic structure in artificial neural networks trained by self-supervision

Manning, C. D., Clark, K., Hewitt, J., Khandelwal, U., & Levy, O. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1907367117 · Cited by 346 · see citing works

Learning How to Ask: Querying LMs with Mixtures of Soft Prompts

Qin, G., & Eisner, J. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.410 · Cited by 342 · see citing works

SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability

Raghu, M., Gilmer, J., Yosinski, J., & Sohl-Dickstein, J. (2017) · arXiv · doi:10.48550/arxiv.1706.05806 · Cited by 335 · see citing works

Probing Classifiers: Promises, Shortcomings, and Advances

Belinkov, Y. (2021) · Computational Linguistics · doi:10.1162/coli_a_00422 · Cited by 311 · see citing works

SKEP: Sentiment Knowledge Enhanced Pre-training for Sentiment Analysis

Tian, H., Gao, C., Xiao, X., Liu, H., He, B., Wu, H. R., Wang, H., & Wu, F. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.374 · Cited by 299 · see citing works

Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification

Hu, S., Ding, N., Wang, H., Liu, Z., Wang, J., Li, J., Wu, W., & Sun, M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.158 · Cited by 298 · see citing works

Factual Probing Is [MASK]: Learning vs. Learning to Recall

Zhong, Z., Friedman, D., & Chen, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.398 · Cited by 274 · see citing works

oLMpics-On What Language Model Pre-training Captures

Talmor, A., Elazar, Y., Goldberg, Y., & Berant, J. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00342 · Cited by 267 · see citing works

Commonsense Knowledge Mining from Pretrained Models

Davison, J., Feldman, J., & Rush, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1109 · Cited by 264 · see citing works

Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly

Kassner, N., & Schütze, H. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.698 · Cited by 259 · see citing works

Zero-Shot Entity Linking by Reading Entity Descriptions

Logeswaran, L., Chang, M.-W., Lee, K., Toutanova, K., Devlin, J., & Lee, H. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1335 · Cited by 254 · see citing works

WARP: Word-level Adversarial ReProgramming

Hambardzumyan, K., Khachatrian, H., & May, J. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.381 · Cited by 243 · see citing works

Open Sesame: Getting inside BERT’s Linguistic Knowledge

Lin, Y., Tan, Y. C., & Frank, R. (2019) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w19-4825 · Cited by 233 · see citing works

A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation

Guan, J., Huang, F., Zhao, Z., Zhu, X., & Huang, M. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00302 · Cited by 230 · see citing works

Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP

Schick, T., Udupa, S., & Schütze, H. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00434 · Cited by 227 · see citing works

Do NLP Models Know Numbers? Probing Numeracy in Embeddings

Wallace, E., Wang, Y., Li, S., Singh, S., & Gardner, M. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1534 · Cited by 227 · see citing works

Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond

Feder, A., Keith, K. A., Manzoor, E., Pryzant, R., Sridhar, D., Wood-Doughty, Z., Eisenstein, J., Grimmer, J., Reichart, R., Roberts, M. E., et al. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00511 · Cited by 220 · see citing works

Measuring and improving consistency in pretrained language models

Elazar, Y., Kassner, N., Ravfogel, S., Ravichander, A., Hovy, E., Schütze, H., & Goldberg, Y. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00410 · Cited by 220 · see citing works

Learning Dense Representations for Entity Retrieval

Gillick, D., Kulkarni, S., Lansing, L., Presta, A., Baldridge, J., Ie, E., & García-Olano, D. (2019) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k19-1049 · Cited by 215 · see citing works

Information-Theoretic Probing with Minimum Description Length

Voita, E., & Titov, I. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.14 · Cited by 197 · see citing works

Knowledge Neurons in Pretrained Transformers

Dai, D., Dong, L., Hao, Y., Sui, Z., Chang, B., & Wei, F. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.581 · Cited by 187 · see citing works

Unsupervised Commonsense Question Answering with Self-Talk

Shwartz, V., West, P., Bras, R. J. L., Bhagavatula, C., & Choi, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.373 · Cited by 186 · see citing works

Editing Factual Knowledge in Language Models

De Cao, N., Aziz, W., & Titov, I. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.522 · Cited by 180 · see citing works

SenseBERT: Driving Some Sense into BERT

Levine, Y., Lenz, B., Dagan, O., Ram, O., Padnos, D., Sharir, O., Shalev‐Shwartz, S., Shashua, A., & Shoham, Y. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.423 · Cited by 179 · see citing works

Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language ModelPreprint

Xiong, W., Du, J., Wang, W. Y., & Stoyanov, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.09637 · Cited by 177 · see citing works

SentiLARE: Sentiment-Aware Language Representation Learning with Linguistic Knowledge

Ke, P., Ji, H., Liu, S., Zhu, X., & Huang, M. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.567 · Cited by 169 · see citing works

Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT

Wu, Z., Chen, Y., Kao, B., & Liu, Q. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.383 · Cited by 159 · see citing works

Evaluating Commonsense in Pre-Trained Language Models

Zhou, X., Zhang, Y., Cui, L., & Huang, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6523 · Cited by 153 · see citing works

Locating and Editing Factual Associations in GPT

Meng, K., Bau, D., Andonian, A., & Belinkov, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1262 · Cited by 149 · see citing works

E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT

Poerner, N., Waltinger, U., & Schütze, H. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.71 · Cited by 146 · see citing works

Untitled work

(2020) · neural information processing systems · Cited by 142 · see citing works

Can language models learn from explanations in context?

Lampinen, A., Dasgupta, I., Chan, S., Mathewson, K., Tessler, M., Creswell, A., McClelland, J., Wang, J., & Hill, F. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.38 · Cited by 142 · see citing works

Entities as Experts: Sparse Memory Access with Entity Supervision

Févry, T., Soares, L. B., FitzGerald, N., Choi, E., & Kwiatkowski, T. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.400 · Cited by 138 · see citing works

Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-Trained Language Models

Lin, B. Y., Lee, S., Khanna, R., & Ren, X. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.557 · Cited by 122 · see citing works

ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning

Qin, Y., Lin, Y., Takanobu, R., Liu, Z., Li, P., Ji, H., Huang, M., Sun, M., & Zhou, J. (2021) · doi:10.18653/v1/2021.acl-long.260 · Cited by 115 · see citing works

WordNet

Miller, G. A. (1992) · DARPA Workshop on Speech and Natural Language (HLT) · doi:10.3115/1075527.1075662 · Cited by 103 · see citing works

Investigating BERT’s Knowledge of Language: Five Analysis Methods with NPIs

Warstadt, A., Cao, Y. H., Grosu, I., Peng, W., Blix, H., Nie, Y., Alsop, A., Bordia, S., Liu, H., Parrish, A., et al. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1286 · Cited by 100 · see citing works

Black-Box Tuning for Language-Model-as-a-Service

Sun, T., Shao, Y., Qian, H., Huang, X., & Qiu, X. (2022) · arXiv · doi:10.48550/arxiv.2201.03514 · Cited by 100 · see citing works

Fast Model Editing at Scale

Mitchell, E., Lin, C., Bosselut, A., Finn, C., & Manning, C. D. (2021) · arXiv · doi:10.48550/arxiv.2110.11309 · Cited by 95 · see citing works

Generate rather than Retrieve: Large Language Models are Strong Context Generators

Yu, W., Iter, D., Wang, S., Xu, Y., Ju, M., Sanyal, S., Zhu, C., Zeng, M., & Jiang, M. (2022) · arXiv · doi:10.48550/arxiv.2209.10063 · Cited by 95 · see citing works

Paradigm Shift in Natural Language Processing

Sun, T., Liu, X.-Y., Qiu, X., & Huang, X. (2022) · Machine Intelligence Research · doi:10.1007/s11633-022-1331-6 · Cited by 94 · see citing works

Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases

Cao, B., Lin, H., Han, X., Sun, L., Yan, L., Liao, M., Xue, T., & Xu, J. (2021) · doi:10.18653/v1/2021.acl-long.146 · Cited by 90 · see citing works

X-FACTR: Multilingual Factual Knowledge Retrieval from Pretrained Language Models

Jiang, Z., Anastasopoulos, A., Araki, J., Ding, H., & Neubig, G. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.479 · Cited by 88 · see citing works

Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models

Kassner, N., Dufter, P., & Schütze, H. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.284 · Cited by 87 · see citing works

Transformers learn in-context by gradient descent

von Oswald, J., Niklasson, E., Randazzo, E., Sacramento, J., Mordvintsev, A., Zhmoginov, A., & Vladymyrov, M. (2022) · arXiv · doi:10.48550/arxiv.2212.07677 · Cited by 87 · see citing works

BERTese: Learning to Speak to BERT

Haviv, A., Berant, J., & Globerson, A. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.316 · Cited by 84 · see citing works

Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees

Bai, J., Wang, Y., Chen, Y., Yang, Y., Bai, J., Yu, J., & Tong, Y. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.262 · Cited by 81 · see citing works

Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NER

Lee, D., Kadakia, A., Tan, K., Agarwal, M., Feng, X., Shibuya, T., Mitani, R., Sekiya, T., Pujara, J., & Ren, X. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.192 · Cited by 80 · see citing works

Language Models are Open Knowledge Graphs

Wang, C., Liu, X., & Song, D. (2020) · arXiv · doi:10.48550/arxiv.2010.11967 · Cited by 79 · see citing works

Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries

Heinzerling, B., & Inui, K. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.153 · Cited by 78 · see citing works

Can Language Models be Biomedical Knowledge Bases?

Sung, M., Lee, J., Yi, S. S., Jeon, M., Kim, S., & Kang, J. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.388 · Cited by 76 · see citing works

Critical Learning Periods in Deep Networks.

Achille, A., Rovere, M., & Soatto, S. (2018) · Cited by 74 · see citing works

Knowledge-driven Data Construction for Zero-shot Evaluation in Commonsense Question Answering

Ma, K., Ilievski, F., Francis, J., Bisk, Y., Nyberg, E., & Oltramari, A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i15.17593 · Cited by 73 · see citing works

Specializing Unsupervised Pretraining Models for Word-Level Semantic Similarity

Lauscher, A., Vulić, I., Ponti, E. M., Korhonen, A., & Glavaš, G. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.118 · Cited by 72 · see citing works

Distributional vectors encode referential attributes

Gupta, A., Boleda, G., Baroni, M., & Padó, S. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1002 · Cited by 70 · see citing works

Selective annotation makes language models better few-shot learners

Su, H., Kasai, J., Wu, C. H., Shi, W., Wang, T., Xin, J., Zhang, R., Ostendorf, M., Zettlemoyer, L., Smith, N. A., et al. (2022) · arXiv · doi:10.48550/arxiv.2209.01975 · Cited by 70 · see citing works

Data Distributional Properties Drive Emergent In-Context Learning in Transformers

Chan, S., Santoro, A., Lampinen, A., Wang, J., Singh, A., Richemond, P., McClelland, J., & Hill, F. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1371 · Cited by 69 · see citing works

Do Syntax Trees Help Pre-trained Transformers Extract Information?

Sachan, D. S., Zhang, Y., Qi, P., & Hamilton, W. L. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.228 · Cited by 68 · see citing works

A Survey of Knowledge Enhanced Pre-trained ModelsPreprint

Yang, J., Xiao, G., Shen, Y.-L., Jiang, W., Hu, X., Zhang, Y., & Peng, J. (2021) · arXiv.org · doi:10.48550/arxiv.2110.00269 · Cited by 64 · see citing works

Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning

Shen, T., Mao, Y., He, P., Long, G., Trischler, A., & Chen, W. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.722 · Cited by 63 · see citing works

A Review on Language Models as Knowledge Bases

AlKhamissi, B., Li, M., Celikyilmaz, A., Diab, M., & Ghazvininejad, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06031 · Cited by 62 · see citing works

What’s in an Embedding? Analyzing Word Embeddings through Multilingual Evaluation

Köhn, A. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1246 · Cited by 61 · see citing works

Align, Mask and Select: A Simple Method for Incorporating Commonsense Knowledge into Language Representation ModelsPreprint

Ye, Z.-X., Chen, Q., Wang, W., & Ling, Z.-H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.06725 · Cited by 61 · see citing works

Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning

Tamborrino, A., Pellicanò, N., Pannier, B., Voitot, P., & Naudin, L. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.357 · Cited by 60 · see citing works

Memory-assisted prompt editing to improve GPT-3 after deployment

Madaan, A., Tandon, N., Clark, P., & Yang, Y. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.183 · Cited by 60 · see citing works

PADA: Example-based Prompt Learning for on-the-fly Adaptation to Unseen Domains

Ben-David, E., Oved, N., & Reichart, R. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00468 · Cited by 54 · see citing works

Self-Supervised Knowledge Triplet Learning for Zero-Shot Question Answering

Banerjee, P., & Baral, C. R. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.11 · Cited by 52 · see citing works

Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models

Finlayson, M., Mueller, A., Gehrmann, S., Shieber, S. M., Linzen, T., & Belinkov, Y. (2021) · doi:10.18653/v1/2021.acl-long.144 · Cited by 52 · see citing works

Memory-Based Model Editing at Scale

Mitchell, E., Lin, C., Bosselut, A., Manning, C. D., & Finn, C. (2022) · arXiv · doi:10.48550/arxiv.2206.06520 · Cited by 51 · see citing works

Editable Neural NetworksPreprint

Sinitsin, A., Plokhotnyuk, V., Pyrkin, D., Popov, S., & Babenko, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.00345 · Cited by 49 · see citing works

Modifying Memories in Transformer Models

Zhu, C., Rawat, A. S., Zaheer, M., Bhojanapalli, S., Li, D., Yu, F., & Kumar, S. (2020) · arXiv · doi:10.48550/arxiv.2012.00363 · Cited by 48 · see citing works

LIMIT-BERT : Linguistics Informed Multi-Task BERT

Zhou, J., Zhang, Z., Zhao, H., & Zhang, S. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.399 · Cited by 45 · see citing works

Probing Across Time: What Does RoBERTa Know and When?

Liu, Z., Wang, Y., Kasai, J., Hajishirzi, H., & Smith, N. A. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.71 · Cited by 42 · see citing works

Explanations from Large Language Models Make Small Reasoners BetterPreprint

Li, S., Chen, J., Shen, Y., Chen, Z., Zhang, X., Li, Z., Wang, H., Jing, Q., Peng, B., Mao, Y., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.06726 · Cited by 39 · see citing works

Calibrating Factual Knowledge in Pretrained Language Models

Dong, Q., Dai, D., Song, Y., Xu, J., Sui, Z., & Li, L. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.438 · Cited by 39 · see citing works

Retrieval-Augmented Multimodal Language ModelingPreprint

Yasunaga, M., Aghajanyan, A., Shi, W., James, R., Leskovec, J., Liang, P., Lewis, M., Zettlemoyer, L., & Yih, W.-T. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.12561 · Cited by 37 · see citing works

A Survey of Knowledge-Intensive NLP with Pre-Trained Language ModelsPreprint

Yin, D., Li, D., Cheng, H., Liu, X., Chang, K.-W., Wei, F., & Gao, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.08772 · Cited by 35 · see citing works

Pretrained Language Model Embryology: The Birth of ALBERT

Chiang, C.-H., Huang, S.-F., & Lee, H.-Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.553 · Cited by 34 · see citing works

How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis

Li, S., Li, X., Shang, L., Dong, Z., Sun, C., Liu, B., Ji, Z., Jiang, X., & Liu, Q. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.136 · Cited by 34 · see citing works

BERTnesia: Investigating the capture and forgetting of knowledge in BERT

Wallat, J., Singh, J., & Anand, A. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.17 · Cited by 33 · see citing works

Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model BeliefsPreprint

Hase, P., Diab, M., Çelikyılmaz, A., Li, X., Kozareva, Z., Stoyanov, V., Bansal, M., & Iyer, S. V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.13654 · Cited by 31 · see citing works

Understanding Learning Dynamics Of Language Models with SVCCA

Saphra, N., & Lopez, A. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1329 · Cited by 31 · see citing works

Probing for Semantic Classes: Diagnosing the Meaning Content of Word Embeddings

Yaghoobzadeh, Y., Kann, K., Hazen, T. J., Agirre, E., & Schütze, H. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1574 · Cited by 30 · see citing works

Can Large Language Models Truly Understand Prompts? A Case Study with Negated PromptsPreprint

Jang, J., Ye, S., & Seo, M. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.12711 · Cited by 30 · see citing works

BLiMP: the benchmark of linguistic minimal pairs for English

Warstadt, A., Parrish, A., Liu, H., Mohananey, A., Peng, W., Wang, S.-F., & Bowman, S. R. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00321 · Cited by 28 · see citing works

How much pretraining data do language models need to learn syntax?

Pérez-Mayos, L., Ballesteros, M., & Wanner, L. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.118 · Cited by 26 · see citing works

Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal View

Cao, B., Lin, H., Han, X., Liu, F., & Sun, L. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.398 · Cited by 22 · see citing works

DirectProbe: Studying Representations without Classifiers

Zhou, Y., & Srikumar, V. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.401 · Cited by 21 · see citing works

Improving Semantic Matching through Dependency-Enhanced Pre-trained Model with Adaptive Fusion

Song, J., Liang, D., Li, R., Li, Y., Wang, S., Peng, M., Wu, W., & Yu, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.4 · Cited by 21 · see citing works

Language Models As or For Knowledge Bases

Razniewski, S., Yates, A., Kassner, N., & Weikum, G. (2021) · arXiv · doi:10.48550/arxiv.2110.04888 · Cited by 16 · see citing works

ThinkSum: Probabilistic reasoning over sets using large language models

Ozturkler, B., Malkin, N., Wang, Z., & Jojić, N. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.68 · Cited by 13 · see citing works

Inducing Relational Knowledge from BERT

Bouraoui, Z., Camacho-Collados, J., & Schockaert, S. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6242 · Cited by 12 · see citing works

LSTMs Compose—and Learn—Bottom-Up

Saphra, N., & Lopez, A. (2020) · doi:10.18653/v1/2020.findings-emnlp.252 · Cited by 10 · see citing works

Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language ModelPreprint

Eisenstein, J., Andor, D., Bohnet, B., Collins, M., & Mimno, D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.02498 · Cited by 10 · see citing works

Robustness of Demonstration-based Learning Under Limited Data Scenario

Zhang, H., Zhang, Y., Zhang, R., & Yang, D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.116 · Cited by 10 · see citing works

Finding patterns in Knowledge Attribution for TransformersPreprint

Juneja, J., & Agarwal, R. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.01366 · Cited by 1 · see citing works

How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances

Zhang, Z., Fang, M., Chen, L., Namazi‐Rad, M., & Wang, Y.-X. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.516 · Cited by 37 · see citing works

The RL/LLM Taxonomy Tree: Reviewing Synergies Between Reinforcement Learning and Large Language Models

Pternea, M., Singh, P., Chakraborty, A., Oruganti, Y. D., Milletari, M., Bapat, S., & Jiang, K. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15960 · Cited by 37 · see citing works

An outline of Prognostics and health management Large Model: Concepts, Paradigms, and challenges

Tao, L., Li, S., Liu, H., Huang, Q., Ma, L., Ning, G., Chen, Y., Wu, Y., Li, B., Zhang, W., et al. (2025) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2025.112683 · Cited by 33 · see citing works

Large Language Models for Recommendation: Progresses and Future Directions

Bao, K., Zhang, J., Zhang, Y., Wenjie, W., Feng, F., & He, X. (2023) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3624918.3629550 · Cited by 31 · see citing works

Large Language Models for Recommendation: Progresses and Future Directions

Zhang, J., Bao, K., Zhang, Y., Wang, W., Feng, F., & He, X. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589335.3641247 · Cited by 20 · see citing works

DocTER: Evaluating document-based knowledge editing

Wu, S., Wang, A., Peng, M., Lin, Y., Li, W., Sun, M., & Su, J. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104299 · Cited by 17 · see citing works

Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons

Chen, Y., Cao, P., Chen, Y., Liu, K., & Zhao, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29735 · Cited by 12 · see citing works

Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging

Shen, L., Tang, A., Yang, E., Guo, G., Luo, Y., Zhang, L., Cao, X., Du, B., & Tao, D. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3629605 · Cited by 10 · see citing works

One mind, many tongues: A deep dive into language-agnostic knowledge neurons in large language models

Cao, P., Chen, Y., Jin, Z., Chen, Y., Liu, K., & Zhao, J. (2026) · Artificial Intelligence · doi:10.1016/j.artint.2026.104490 · Cited by 6 · see citing works

Matching tasks to objectives: Fine-tuning and prompt-tuning strategies for encoder-decoder pre-trained language models

Pouramini, A., & Faili, H. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05660-2 · Cited by 6 · see citing works

Large Language Models as Biomedical Hypothesis Generators: A Comprehensive Evaluation

Qi, B., Zhang, K., Tian, K., Li, H., Chen, Z.-R., Zeng, S., Hua, E., Jinfang, H., & Zhou, B. (2024) · arXiv · doi:10.48550/arxiv.2407.08940 · Cited by 5 · see citing works

Owl-vision-inspired near sensor computing

Zhao, Z., Cao, Y., Huang, S., Fan, K., Ding, Y., Zhu, G., Li, L., Liu, Q., Deng, W., Zhao, C., et al. (2026) · Nature Communications · doi:10.1038/s41467-026-69123-7 · Cited by 4 · see citing works

Software Development Process Model for Machine Learning and Evolving Variants: A Systematic Literature Review

Jamal, S., Ahmad, A., & ZESHAN, D. F. (2026) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194026300010 · Cited by 1 · see citing works

We Live in Interesting Times: Introduction to the Special Section on Big Data & Behavior Science

Cox, D. J., & Young, M. E. (2024) · Perspectives on Behavior Science · doi:10.1007/s40614-024-00400-w · Cited by 1 · see citing works

Does the Correctness of Factual Knowledge Matter for Factual Knowledge-Enhanced Pre-trained Language Models?

Cao, B., Tang, Q., Lin, H., Han, X., & Sun, L. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.143 · Cited by 1 · see citing works

BEAR: A Unified Framework for Evaluating Relational Knowledge in Causal and Masked Language Models

Wiland, J., Ploner, M., & Akbik, A. (2024) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2024.findings-naacl.155 · Cited by 1 · see citing works

SJ-ORPO: Alignment of LLMs Using Multi-Task Learning and Self-Judgment

Parashar, J., & Bhandarkar, S. M. (2025) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai64502.2025.00040

62120 Semantic Web

Ansgar Scherp, A., & Gröner, G. (2026) · doi:10.1515/9783111386690-024

Reformulation is All You Need: Addressing Malicious Textual Features in DNNs

Jiang, Y., Ma, O., Yang, Y., Zhang, T., & Ji, S. (2026) · Machine Intelligence Research · doi:10.1007/s11633-025-1594-9

Countering Adversarial Attacks with Multimodal Image Fusion

Dong, Y., Zhang, C., Zhang, X., Zhang, G., & Zhao, Y. (2026) · Machine Intelligence Research · doi:10.1007/s11633-025-1613-x

Transformer-LLM Smart Classroom Assessment System: Sequence Labeling and Multi-Task Learning Methods Oriented to Ideological and Political Education in Curriculum

Han, X., Lin, Y., & Shang, P. (2026) · International Conference on Artificial Intelligence in Education Technology and Data Science · doi:10.1145/3810158.3810182

Continual Egocentric Activity Recognition by Multi-level Multimodal Feature Calibration and Temporal Aggregation

Dai, Y., Qiu, H., Wang, L., Qiu, B., Zhao, T., Meng, F., & Li, H. (2026) · Machine Intelligence Research · doi:10.1007/s11633-025-1604-y

A Single Frozen Transformer in Language Models is Simple yet Powerful Time Series Forecaster

Zhang, P., Li, Z., Gao, Z., & Xiang, S. (2026) · Machine Intelligence Research · doi:10.1007/s11633-025-1622-x

Knowledge Enhanced Pretrained Language Models: A Compreshensive SurveyPreprint

Wei, X., Wang, S., Zhang, D., Bhatia, P., & Arnold, A. O. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08455 · Cited by 38 · see citing works

Measuring the Knowledge Acquisition-Utilization Gap in Pretrained Language Models

Kazemnejad, A., Rezagholizadeh, M., Parthasarathi, P., & Chandar, S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.285 · Cited by 4 · see citing works

A Survey on Knowledge-Enhanced Pre-trained Language ModelsPreprint

Zhen, C., Shang, Y., Liu, X., Li, Y., Chen, Y., & Zhang, D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2212.13428 · Cited by 11 · see citing works

Knowledge Rumination for Pre-trained Language Models

Yao, Y., Wang, P., Mao, S., Tan, C., Huang, F., Chen, H., & Zhang, N. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.206 · Cited by 5 · see citing works

Knowledge Inheritance for Pre-trained Language ModelsPreprint

Qin, Y., Lin, Y., Yi, J., Zhang, J., Han, X., Zhang, Z., Su, Y., Liu, Z., Li, P., Sun, M., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.13880 · Cited by 12 · see citing works