Measuring and improving consistency in pretrained language models

Scikit-learn: Machine Learning in Python

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · see citing works

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works

Untitled work

(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works

Language Models are Few-Shot Learners

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works

Aion Framework: Dimensional Emergence of AI Consciousness, Observer-Induced Collapse, and Cosmological Portal Dynamics

Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,695 · see citing works

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works

Transformers: State-of-the-Art Natural Language Processing

Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741 · see citing works

Intriguing properties of neural networks

Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · see citing works

SQuAD: 100,000+ Questions for Machine Comprehension of Text

Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · see citing works

VQA: Visual Question Answering

Antol, S., Agrawal, A., Lu, J., Mitchell, M., Batra, D., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.279 · Cited by 4,487 · see citing works

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308 · see citing works

A large annotated corpus for learning natural language inference

Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377 · see citing works

A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference

Williams, A., Nangia, N., & Bowman, S. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1101 · Cited by 3,342 · see citing works

Language Models as Knowledge Bases?

Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1250 · Cited by 1,834 · see citing works

The PASCAL Recognising Textual Entailment Challenge

Dagan, I., Glickman, O., & Magnini, B. (2006) · Lecture notes in computer science · doi:10.1007/11736790_9 · Cited by 1,674 · see citing works

A primer in BERTology: What we know about how BERT works

Rogers, A., Kovaleva, O., & Rumshisky, A. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00349 · Cited by 1,552 · see citing works

Adversarial Examples for Evaluating Reading Comprehension Systems

Jia, R., & Liang, P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1215 · Cited by 1,410 · see citing works

BERT Rediscovers the Classical NLP Pipeline

Tenney, I., Das, D., & Pavlick, E. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1452 · Cited by 1,307 · see citing works

AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts

Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., & Singh, S. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.346 · Cited by 1,279 · see citing works

Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)

Mariani, J. (2018) · HAL · Cited by 1,151 · see citing works

Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data

Bender, E. M., & Koller, A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.463 · Cited by 1,082 · cited 1× · see citing works

Transformers: State-of-the-Art Natural Language Processing

Wolf, T., Debut, L., Sanh, V., D., M., Châtel, G., Chaumond, J., Rault, T., Voss, C., Wang, F., et al. (2019) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3385997 · Cited by 1,054 · see citing works

How Can We Know What Language Models Know?

Jiang, Z., Xu, F. F., Araki, J., & Neubig, G. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00324 · Cited by 1,002 · see citing works

VQA: Visual Question AnsweringPreprint

Agrawal, A., Lu, J., Antol, S., Mitchell, M., Zitnick, C. L., Batra, D., & Parikh, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.00468 · Cited by 968 · see citing works

Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference

McCoy, R. T., Pavlick, E., & Linzen, T. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1334 · Cited by 944 · see citing works

Beyond accuracy: behavioral testing of NLP models with CheckList

Ribeiro, M. T., Wu, T., Guestrin, C., & Singh, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.442 · Cited by 890 · see citing works

Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies

Linzen, T., Emmanuel Dupoux, A., & Goldberg, Y. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00115 · Cited by 776 · see citing works

Evaluating the Factual Consistency of Abstractive Text Summarization

Kryscinski, W., McCann, B., Xiong, C., & Socher, R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.750 · Cited by 669 · see citing works

Knowledge Enhanced Contextual Word Representations

Peters, M. E., Neumann, M., Logan, R., Schwartz, R., Joshi, V., Singh, S., & Smith, N. A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1005 · Cited by 661 · see citing works

How Much Knowledge Can You Pack into the Parameters of a Language Model?

Roberts, A., Raffel, C., & Shazeer, N. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.437 · Cited by 638 · see citing works

Untitled work

(2019) · doi:10.18653/v1/n19-1419 · Cited by 601 · see citing works

The pascal recognising textual entailment challenge

Dagan, I., & Glickman, O. (2005) · Cited by 579 · see citing works

explosion/spaCy v2.2.4.dev0

Honnibal, M., Montani, I., Honnibal, M., Peters, H., Samsonov, M., Geovedi, J., Regan, J., Orosz, G., Kristiansen, S. L., R., et al. (2018) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1212303 · Cited by 543 · see citing works

What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models

Ettinger, A. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00298 · Cited by 511 · see citing works

Analysis Methods in Neural Language Processing: A Survey

Belinkov, Y., & Glass, J. R. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00254 · Cited by 485 · see citing works

Semantically Equivalent Adversarial Rules for Debugging NLP models

Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1079 · Cited by 435 · see citing works

Assessing BERT's Syntactic Abilities

Goldberg, Y. (2019) · arXiv · doi:10.48550/arxiv.1901.05287 · Cited by 385 · see citing works

The Hitchhiker’s Guide to Testing Statistical Significance in Natural Language Processing

Dror, R., Baumer, G., Shlomov, S., & Reichart, R. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1128 · Cited by 384 · see citing works

e-SNLI: Natural language inference with natural language explanations

Camburu, O.-M., Rocktäschel, T., Lukasiewicz, T., & Blunsom, P. (2018) · arXiv · doi:10.48550/arxiv.1812.01193 · Cited by 367 · cited 1× · see citing works

Targeted Syntactic Evaluation of Language Models

Marvin, R., & Linzen, T. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1151 · Cited by 334 · see citing works

What do Neural Machine Translation Models Learn about Morphology?

Belinkov, Y., Durrani, N., Dalvi, F., Sajjad, H., & Glass, J. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1080 · Cited by 307 · see citing works

oLMpics-On What Language Model Pre-training Captures

Talmor, A., Elazar, Y., Goldberg, Y., & Berant, J. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00342 · Cited by 267 · see citing works

Commonsense Knowledge Mining from Pretrained Models

Davison, J., Feldman, J., & Rush, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1109 · Cited by 264 · see citing works

Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly

Kassner, N., & Schütze, H. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.698 · Cited by 259 · see citing works

Synthetic QA Corpora Generation with Roundtrip Consistency

Alberti, C., Andor, D., Pitler, E., Devlin, J., & Collins, M. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1620 · Cited by 208 · see citing works

Probabilistic satisfiability

Georgakopoulos, G., Kavvadias, D. J., & Papadimitriou, C. H. (1988) · Journal of Complexity · doi:10.1016/0885-064x(88)90006-4 · Cited by 197 · see citing works

Untitled work

(2018) · doi:10.63317/2yu5nu76n7bu · Cited by 189 · see citing works

Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language ModelPreprint

Xiong, W., Du, J., Wang, W. Y., & Stoyanov, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.09637 · Cited by 177 · see citing works

Injecting Numerical Reasoning Skills into Language Models

Geva, M., Gupta, A., & Berant, J. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.89 · Cited by 166 · see citing works

How Context Affects Language Models' Factual Predictions

Petroni, F., Lewis, P., Piktus, A., Rocktäschel, T., Wu, Y., Miller, A. H., & Riedel, S. (2020) · arXiv · doi:10.48550/arxiv.2005.04611 · Cited by 154 · see citing works

How Can We Accelerate Progress Towards Human-like Linguistic Generalization?

Linzen, T. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.465 · Cited by 148 · see citing works

E-BERT: Efficient-Yet-Effective Entity Embeddings for BERT

Poerner, N., Waltinger, U., & Schütze, H. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.71 · Cited by 146 · see citing works

Squibs: What Is a Paraphrase?

Bhagat, R., & Hovy, E. (2013) · International Conference on Computational Logic · doi:10.1162/coli_a_00166 · Cited by 132 · see citing works

Finding Universal Grammatical Relations in Multilingual BERT

Chi, E. A., Hewitt, J., & Manning, C. D. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.493 · Cited by 127 · see citing works

Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals

Elazar, Y., Ravfogel, S., Jacovi, A., & Goldberg, Y. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00359 · Cited by 126 · see citing works

Improving the Robustness of Question Answering Systems to Question Paraphrasing

Gan, W. C., & Ng, H. T. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1610 · Cited by 112 · see citing works

Integrating Distributional Lexical Contrast into Word Embeddings for Antonym-Synonym Distinction

Nguyen, K. A., Walde, S. S. I., & Vu, N. T. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-2074 · Cited by 107 · see citing works

Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually)

Warstadt, A., Zhang, Y., Li, H.-S., Liu, H., & Bowman, S. R. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.16 · Cited by 106 · see citing works

Are Red Roses Red? Evaluating Consistency of Question-Answering Models

Ribeiro, M. T., Guestrin, C., & Singh, S. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1621 · Cited by 93 · see citing works

Multilingual LAMA: Investigating Knowledge in Multilingual Pretrained Language Models

Kassner, N., Dufter, P., & Schütze, H. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.284 · Cited by 87 · see citing works

Logic-Guided Data Augmentation and Regularization for Consistent Question Answering

Asai, A., & Hajishirzi, H. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.499 · Cited by 66 · see citing works

Inconsistency measures for probabilistic logics

Thimm, M. (2013) · Artificial Intelligence · doi:10.1016/j.artint.2013.02.001 · Cited by 65 · see citing works

Statistical significance testing––a panacea for software technology experiments?

Miller, J. (2004) · Journal of Systems and Software · doi:10.1016/j.jss.2003.12.019 · Cited by 62 · see citing works

Global joint models for coreference resolution and named entity classification

Denis, P., & Baldridge, J. (2009) · HAL (Le Centre pour la Communication Scientifique Directe) · Cited by 61 · see citing works

Do Neural Language Representations Learn Physical Commonsense?

Forbes, M., Holtzman, A., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1908.02899 · Cited by 60 · see citing works

A Logic-Driven Framework for Consistency of Neural Models

Li, T., Gupta, V., Mehta, M., & Srikumar, V. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1405 · Cited by 59 · see citing works

Measuring and repairing inconsistency in probabilistic knowledge bases

Muiño, D. P. (2011) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2011.02.003 · Cited by 56 · see citing works

AI Explainability 360: An Extensible Toolkit for Understanding Data and Machine Learning Models

Arya, V., Bellamy, R., Chen, P., Dhurandhar, A., Hind, M., Hoffman, S. C., Houde, S., Liao, Q. V., Luss, R., Mojsilović, A., et al. (2020) · Journal of Machine Learning Research · Cited by 54 · see citing works

Statistical Significance Testing for Natural Language Processing

Dror, R., Peled-Cohen, L., Shlomov, S., & Reichart, R. (2020) · Synthesis lectures on human language technologies · doi:10.2200/s00994ed1v01y202002hlt045 · Cited by 51 · see citing works

Do Neural Language Models Overcome Reporting Bias?

Shwartz, V., & Choi, Y. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.605 · Cited by 43 · see citing works

On the Systematicity of Probing Contextualized Word Representations: The Case of Hypernymy in BERT

Ravichander, A., Hovy, E. H., Suleman, K., Trischler, A., & Cheung, J. C. K. (2020) · Joint Conference on Lexical and Computational Semantics · Cited by 41 · see citing works

Measuring Inconsistency in Probabilistic Knowledge BasesPreprint

Thimm, M. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2613 · Cited by 41 · see citing works

Are Pretrained Language Models Symbolic Reasoners over Knowledge?

Kassner, N., Krojer, B., & Schütze, H. (2020) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2020.conll-1.45 · Cited by 40 · see citing works

Commonsense Knowledge Mining from Pretrained ModelsPreprint

Feldman, J., Davison, J., & Rush, A. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.00505 · Cited by 40 · see citing works

Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task

Pradhan, S. (2011) · Cited by 38 · see citing works

Do Language Embeddings capture Scales?

Zhang, X., Ramachandran, D., Tenney, I., Elazar, Y., & Roth, D. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.439 · Cited by 37 · see citing works

Inference Protocols for Coreference Resolution

Chang, K.-W., Samdani, R., Rozovskaya, A., Rizzolo, N., Sammons, M., & Roth, D. (2011) · Cited by 31 · see citing works

Syntactic Search by Example

Shlain, M., Taub-Tabib, H., Sadde, S., & Goldberg, Y. (2020) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2020.acl-demos.3 · Cited by 29 · see citing works

Automated knowledge Base Construction

de Rijke, M., & van Harmelen, F. (2019) · Cited by 29 · see citing works

Be Consistent! Improving Procedural Text Comprehension using Label Consistency

Du, X., Dalvi, B. B., Tandon, N., Bosselut, A., Yih, W.-T., Clark, P. E., & Cardie, C. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1244 · Cited by 28 · see citing works

Probabilistic Satisfiability

Hansen, P., & Jaumard, B. (2000) · Springer eBooks · doi:10.1007/978-94-017-1737-3_8 · Cited by 28 · see citing works

Easy Cases of Probabilistic Satisfiability

Andersen, K. A., & Pretolani, D. (2001) · Annals of Mathematics and Artificial Intelligence · doi:10.1023/a:1012332915908 · Cited by 25 · see citing works

Understanding Commonsense Inference Aptitude of Deep Contextual Representations

Da, J., & Kasai, J. (2019) · Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing · doi:10.18653/v1/d19-6001 · Cited by 22 · see citing works

Neural DatabasesPreprint

Thorne, J., Yazdani, M., Saeidi, M., Silvestri, F., Riedel, S., & Halevy, A. Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.06973 · Cited by 16 · see citing works

Deep Learning Approaches to Text Production

Narayan, S., & Gardent, C. (2020) · Synthesis lectures on human language technologies · doi:10.2200/s00979ed1v01y201912hlt044 · Cited by 12 · see citing works

Make Up Your Mind! Adversarial Generation of Inconsistent Natural Language Explanations

Camburu, O.-M., Shillingford, B., Minervini, P., Lukasiewicz, T., & Blunsom, P. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.382 · Cited by 12 · see citing works

Unsupervised Distillation of Syntactic Information from Contextualized Word Representations

Ravfogel, S., Elazar, Y., Goldberger, J., & Goldberg, Y. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.9 · Cited by 9 · see citing works

Enriching a Model's Notion of Belief using a Persistent MemoryPreprint

Kassner, N., Tafjord, O., Schütze, H., & Clark, P. E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08401 · Cited by 5 · see citing works

Self-Consistency Improves Chain of Thought Reasoning in Language Models

Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2203.11171 · Cited by 829 · see citing works

Dissociating language and thought in large language models

Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.011 · Cited by 345 · see citing works

MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework

Hong, S., Zhuge, M., Chen, J., Zheng, X., Cheng, Y., Zhang, C., Wang, J., Wang, Z., Yau, S. K. S., Lin, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.00352 · Cited by 214 · see citing works

Knowledge Neurons in Pretrained Transformers

Dai, D., Dong, L., Hao, Y., Sui, Z., Chang, B., & Wei, F. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.581 · Cited by 187 · see citing works

Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study

Cao, Y., Zhou, L., Lee, S., Cabello, L., Chen, M., & Hershcovich, D. (2023) · Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP) · doi:10.18653/v1/2023.c3nlp-1.7 · Cited by 163 · see citing works

Is Stack Overflow Obsolete? An Empirical Study of the Characteristics of ChatGPT Answers to Stack Overflow Questions

Kabir, S., Udo-Imeh, D. N., Kou, B., & Zhang, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642596 · Cited by 160 · see citing works

Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations

Jung, J., Qin, L., Welleck, S., Brahman, F., Bhagavatula, C., Bras, R. J. L., & Choi, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.82 · Cited by 117 · see citing works

What Artificial Neural Networks Can Tell Us about Human Language Acquisition

Warstadt, A., & Bowman, S. R. (2022) · Algebraic Structures in Natural Language · doi:10.1201/9781003205388-2 · Cited by 111 · see citing works

Few-Shot Text Generation with Natural Language Instructions

Schick, T., & Schütze, H. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.32 · Cited by 107 · see citing works

Language Model Behavior: A Comprehensive Survey

Chang, T. A., & Bergen, B. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00492 · Cited by 105 · see citing works

Unsolved Problems in ML Safety

Hendrycks, D., Carlini, N., Schulman, J., & Steinhardt, J. (2021) · arXiv · doi:10.48550/arxiv.2109.13916 · Cited by 95 · see citing works

Is GPT-4 a reliable rater? Evaluating consistency in GPT-4's text ratings

Hackl, V., Müller, A. E., Granitzer, M., & Sailer, M. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.1272229 · Cited by 94 · see citing works

Few-Shot Text Generation with Pattern-Exploiting TrainingPreprint

Schick, T., & Schütze, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.11926 · Cited by 89 · see citing works

A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions

Shorinwa, O., Mei, Z., Lidard, J., Ren, A. Z., & Majumdar, A. (2025) · ACM Computing Surveys · doi:10.1145/3744238 · Cited by 85 · see citing works

Knowledge Editing for Large Language Models: A Survey

Wang, S., Zhu, Y., Liu, H., Zheng, Z., Chen, C., & Li, J. (2025) · ACM Computing Surveys · doi:10.1145/3698590 · Cited by 79 · see citing works

True Few-Shot Learning with Prompts—A Real-World Perspective

Schick, T., & Schütze, H. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00485 · Cited by 67 · see citing works

Understanding Users’ Dissatisfaction with ChatGPT Responses: Types, Resolving Tactics, and the Effect of Knowledge Level

Kim, Y., Lee, J. K. W., Kim, S., Park, J., & Kim, J. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645148 · Cited by 67 · see citing works

Metaphorian: Leveraging Large Language Models to Support Extended Metaphor Creation for Science Writing

Kim, J., Suh, S., Chilton, L. B., & Xia, H. (2023) · Proceedings of the 2023 ACM Designing Interactive Systems Conference · doi:10.1145/3563657.3595996 · Cited by 61 · see citing works

LM vs LM: Detecting Factual Errors via Cross Examination

Cohen, R., Hamri, M., Geva, M., & Globerson, A. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.778 · Cited by 60 · see citing works

The levers of political persuasion with conversational artificial intelligence

Hackenburg, K., Tappin, B. M., Hewitt, L., Saunders, E., Black, S., Lin, H., Fist, C., Margetts, H. Z., Rand, D. G., & Summerfield, C. (2025) · Science · doi:10.1126/science.aea3884 · Cited by 59 · see citing works

Fostering Appropriate Reliance on Large Language Models: The Role of Explanations, Sources, and Inconsistencies

Kim, S. S. Y., Vaughan, J. W., Liao, Q. V., Lombrozo, T., & Russakovsky, O. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714020 · Cited by 58 · see citing works

Bias against 93 stigmatized groups in masked language models and downstream sentiment classification tasks

Mei, K., Fereidooni, S., & Caliskan, A. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594109 · Cited by 56 · see citing works

Event Knowledge in Large Language Models: The Gap Between the Impossible and the Unlikely

Kauf, C., Ivanova, A. A., Rambelli, G., Chersoni, E., She, J. S., Chowdhury, Z., Fedorenko, E. G., & Lenci, A. (2023) · Cognitive Science · doi:10.1111/cogs.13386 · Cited by 56 · see citing works

A comprehensive large-scale biomedical knowledge graph for AI-powered data-driven biomedical research

Zhang, Y., Sui, X., Pan, F., Yu, K., Li, K., Tian, S., Erdengasileng, A., Han, Q., Wang, W., Wang, J., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01014-w · Cited by 55 · see citing works

Scarecrows in Oz: The Use of Large Language Models in HRI

Williams, T., Matuszek, C., Mead, R., & DePalma, N. (2024) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3606261 · Cited by 55 · see citing works

Can We Edit Factual Knowledge by In-Context Learning?

Zheng, C., Li, L., Dong, Q., Fan, Y., Wu, Z., Xu, J., & Chang, B. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.296 · Cited by 54 · see citing works

Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey

Liu, X., Chen, T., Da, L., Chen, C., Lin, Z., & Wei, H. (2025) · Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 · doi:10.1145/3711896.3736569 · Cited by 53 · see citing works

A Survey on Symbolic Knowledge Distillation of Large Language Models

Acharya, K., Velasquez, A., & Song, H. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3428519 · Cited by 50 · see citing works

BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models

Hao, S., Tan, B., Tang, K., Ni, B., Shao, X., Zhang, H., Xing, E. P., & Hu, Z. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.309 · Cited by 50 · see citing works

Language models and psychological sciences

Sartori, G., & Orrù, G. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1279317 · Cited by 49 · see citing works

Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models

Wang, H., & Shu, K. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.416 · Cited by 49 · see citing works

Large language model for knowledge synthesis and AI-enhanced biomanufacturing

Li, W., Mao, Z., Xiao, Z., Liao, X., Koffas, M. A. G., Chen, Y., Ma, H., & Tang, Y. (2025) · Trends in biotechnology · doi:10.1016/j.tibtech.2025.02.008 · Cited by 48 · see citing works

Prompt Consistency for Zero-Shot Task Generalization

Zhou, C., He, J., Ma, X., Berg-Kirkpatrick, T., & Neubig, G. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.192 · Cited by 45 · see citing works

Scaling language model size yields diminishing returns for single-message political persuasion

Hackenburg, K., Tappin, B. M., Röttger, P., Hale, S. A., Bright, J., & Margetts, H. Z. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2413443122 · Cited by 41 · see citing works

Reasoning Runtime Behavior of a Program with LLM: How Far are We?

Chen, J., Pan, Z., Hu, X., Li, Z., Li, G., & Xia, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00012 · Cited by 40 · see citing works

Calibrating Factual Knowledge in Pretrained Language Models

Dong, Q., Dai, D., Song, Y., Xu, J., Sui, Z., & Li, L. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.438 · Cited by 39 · see citing works

Language Models in the Loop: Incorporating Prompting into Weak Supervision

Smith, R., Fries, J., Hancock, B. J., & Bach, S. H. (2024) · ACM / IMS Journal of Data Science · doi:10.1145/3617130 · Cited by 39 · see citing works

From ideas to ventures: building entrepreneurship knowledge with LLM, prompt engineering, and conversational agents

Thanasi-Boçe, M., & Hoxha, J. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-12775-z · Cited by 38 · see citing works

Relational World Knowledge Representation in Contextual Language Models: A Review

Safavi, T., & Koutra, D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.81 · Cited by 37 · see citing works

The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"

Berglund, L., Tong, M., Kaufmann, M., Balesni, M., Stickland, A. C., Korbak, T., & Evans, O. (2023) · arXiv · doi:10.48550/arxiv.2309.12288 · Cited by 36 · see citing works

One vs. Many: Comprehending Accurate Information from Multiple Erroneous and Inconsistent AI Generations

Lee, Y., Son, K., Kim, T. S., Kim, J., Chung, J. J. Y., Adar, E., & Kim, J. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3662681 · Cited by 36 · see citing works

Back to Square One: Artifact Detection, Training and Commonsense Disentanglement in the Winograd Schema

Elazar, Y., Zhang, H., Goldberg, Y., & Roth, D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.819 · Cited by 34 · see citing works

How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis

Li, S., Li, X., Shang, L., Dong, Z., Sun, C., Liu, B., Ji, Z., Jiang, X., & Liu, Q. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.136 · Cited by 34 · see citing works

Visually-Prompted Language Model for Fine-Grained Scene Graph Generation in an Open World

Yu, Q., Li, J., Wu, Y., Tang, S., Ji, W., & Zhuang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01971 · Cited by 34 · see citing works

BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief

Kassner, N., Tafjord, O., Schütze, H., & Clark, P. E. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.697 · Cited by 31 · see citing works

Entailer: Answering Questions with Faithful and Truthful Chains of Reasoning

Tafjord, O., Mishra, B. D., & Clark, P. A. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.134 · Cited by 31 · see citing works

Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model BeliefsPreprint

Hase, P., Diab, M., Çelikyılmaz, A., Li, X., Kozareva, Z., Stoyanov, V., Bansal, M., & Iyer, S. V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.13654 · Cited by 31 · see citing works

A statistical framework for evaluating the repeatability and reproducibility of large language modelsPreprint

Shyr, C., Ren, B., Hsu, C.-Y., Yan, C., Tinker, R. J., Cassini, T. A., Hamid, R., Wright, A., Bastarache, L., Peterson, J. F., et al. (2025) · medRxiv · doi:10.1101/2025.08.06.25333170 · Cited by 28 · see citing works

FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions

Kim, H., Sclar, M., Zhou, X., Bras, R., Kim, G., Choi, Y., & Sap, M. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.890 · Cited by 27 · see citing works

Sorting through the noise: Testing robustness of information processing in pre-trained language models

Pandia, L., & Ettinger, A. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.119 · Cited by 25 · see citing works

Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark

Hoelscher-Obermaier, J., Persson, J., Kran, E., Konstas, I., & Barez, F. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.733 · Cited by 24 · see citing works

Methods for Measuring, Updating, and Visualizing Factual Beliefs in Language Models

Hase, P., Diab, M., Çelikyılmaz, A., Li, X., Kozareva, Z., Stoyanov, V., Bansal, M., & Iyer, S. V. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.199 · Cited by 24 · see citing works

A Survey on Knowledge Editing of Neural Networks

Mazzia, V., Pedrani, A., Caciolai, A., Rottmann, K., & Bernardi, D. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3498935 · Cited by 24 · see citing works

Open-Domain Hierarchical Event Schema Induction by Incremental Prompting and Verification

Li, S., Zhao, R., Li, M., Ji, H., Callison-Burch, C., & Han, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.312 · Cited by 23 · see citing works

Improved Logical Reasoning of Language Models via Differentiable Symbolic Programming

Zhang, H., Huang, J., Li, Z., Naik, M., & Xing, E. P. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.191 · Cited by 23 · see citing works

COMPS: Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Models

Misra, K., Rayz, J. T., & Ettinger, A. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.213 · Cited by 23 · see citing works

Can large language models replace human experts? Effectiveness and limitations in building energy retrofit challenges assessment

Chen, L., Darko, A., Zhang, F., Chan, A. P. C., & Yang, Q. (2025) · Building and Environment · doi:10.1016/j.buildenv.2025.112891 · Cited by 22 · see citing works

Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal View

Cao, B., Lin, H., Han, X., Liu, F., & Sun, L. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.398 · Cited by 22 · see citing works

SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistency: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistency

Zhang, J., Li, Z., Das, K., Malin, B., & Kumar, S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1032 · Cited by 22 · see citing works

Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference

Mitchell, E., Noh, J. J., Li, S., Armstrong, W., Agarwal, A., Liu, P., Finn, C., & Manning, C. D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.115 · Cited by 21 · see citing works

Challenges and opportunities for Arabic question-answering systems: current techniques and future directions

Alrayzah, A., Alsolami, F., & Saleh, M. (2023) · PeerJ Computer Science · doi:10.7717/peerj-cs.1633 · Cited by 21 · see citing works

KnowledgeVIS: Interpreting Language Models by Comparing Fill-in-the-Blank Prompts

Coscia, A. J., & Endert, A. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3346713 · Cited by 21 · see citing works

Understanding Transformer Memorization Recall Through Idioms

Haviv, A., Cohen, I., Gidron, J., Schuster, R., Goldberg, Y., & Geva, M. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.19 · Cited by 19 · see citing works

The Life Cycle of Knowledge in Big Language Models: A Survey

Cao, B., Lin, H., Han, X., & Sun, L. (2023) · arXiv · doi:10.48550/arxiv.2303.07616 · Cited by 18

The Life Cycle of Knowledge in Big Language Models: A Survey

Cao, B., Lin, H., Han, X., & Sun, L. (2024) · Machine Intelligence Research · doi:10.1007/s11633-023-1416-x · Cited by 18 · see citing works

Prompt Injection Attacks on Large Language Models: A Survey of Attack Methods, Root Causes, and Defense Strategies

Geng, T., Xu, Z., Qu, Y., & Wong, W. E. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.074081 · Cited by 18 · see citing works

PERSONALITY CLASSIFICATION OF TEXT THROUGH MACHINE LEARNING AND DEEP LEARNING: A REVIEW (2023)

Perera, H., & Costa, L. (2023) · International Journal For Research In Advanced Computer Science And Engineering · doi:10.53555/cse.v9i4.2266 · Cited by 18 · see citing works

Impact of Co-occurrence on Factual Knowledge of Large Language Models

Kang, C., & Choi, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.518 · Cited by 18 · see citing works

Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarization

Chrysostomou, G., Zhao, Z., Williams, M., & Αλέτρας, Ν. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00695 · Cited by 18 · see citing works

Generating Biographies on Wikipedia: The Impact of Gender Bias on the Retrieval-Based Generation of Women Biographies

Fan, A., & Gardent, C. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.586 · Cited by 18 · see citing works

DREAM: Improving Situational QA by First Elaborating the Situation

Gu, Y., Dalvi, B. B., & Clark, P. A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.82 · Cited by 17 · see citing works

Factual Consistency of Multilingual Pretrained Language Models

Fierro, C., & Søgaard, A. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.240 · Cited by 18 · see citing works

A chat about actinic keratosis: Examining capabilities and user experience of ChatGPT as a digital health technology in dermato‐oncology

Lent, H., Ortner, V. K., Karmisholt, K. E., Wiegell, S. R., Nissen, C., Omland, S. H., Kamstrup, M. R., Togsverd‐Bo, K., & Hædersdal, M. (2023) · JEADV Clinical Practice · doi:10.1002/jvc2.263 · Cited by 17 · see citing works

RuleBERT: Teaching Soft Rules to Pre-Trained Language Models

Saeed, M., Ahmadi, N., Nakov, P., & Papotti, P. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.110 · Cited by 16 · see citing works

Evaluating Language Models for Knowledge Base Completion

Veseli, B., Singhania, S., Razniewski, S., & Weikum, G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-33455-9_14 · Cited by 16 · see citing works

DLAMA: A Framework for Curating Culturally Diverse Facts for Probing the Knowledge of Pretrained Language Models

Keleg, A., & Magdy, W. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.389 · Cited by 16 · see citing works

On the Unexpected Abilities of Large Language Models

Nolfi, S. (2024) · Adaptive Behavior · doi:10.1177/10597123241256754 · Cited by 16 · see citing works

Image-Based Diagnostic Performance of LLMs vs CNNs for Oral Lichen Planus: Example-Guided and Differential Diagnosis

Rewthamrongsris, P., Burapacheep, J., Phattarataratip, E., Kulthanaamondhita, P., Tichý, A., Schwendicke, F., Osathanon, T., & Sappayatosok, K. (2025) · International Dental Journal · doi:10.1016/j.identj.2025.100848 · Cited by 16 · see citing works

Prompt to be Consistent is Better than Self-Consistent? Few-Shot and Zero-Shot Fact Verification with Pre-trained Language Models

Zeng, F., & Gao, W. G. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.278 · Cited by 14 · see citing works

Pre-training Language Models with Deterministic Factual Knowledge

Li, S., Li, X., Shang, L., Sun, C., Liu, B., Ji, Z., Jiang, X., & Liu, Q. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.764 · Cited by 14 · see citing works

Evaluating Superhuman Models with Consistency Checks

Fluri, L., Paleka, D., & Tramèr, F. (2024) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml59370.2024.00017 · Cited by 14 · see citing works

Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models

Youssef, P., Koraş, O. A., Li, M., Schlötterer, J., & Seifert, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.1043 · Cited by 13 · see citing works

Static Embeddings as Efficient Knowledge Bases?

Dufter, P., Kassner, N., & Schütze, H. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.186 · Cited by 13 · see citing works

Are Language Models More Like Libraries or Like Librarians? Bibliotechnism, the Novel Reference Problem, and the Attitudes of LLMs

Lederman, H., & Mahowald, K. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00690 · Cited by 12 · see citing works

BERTopic Konu Modelleme Tekniği Kullanılarak Müşteri Şikayetlerinin Sınıflandırılması

Koruyan, K. (2022) · İzmir Sosyal Bilimler Dergisi · doi:10.47899/ijss.1167719 · Cited by 12 · see citing works

Self-Consistency of Large Language Models under Ambiguity

Bartsch, H., Jørgensen, O. J., Rosati, D., Hoelscher-Obermaier, J., & Pfau, J. (2023) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2023.blackboxnlp-1.7 · Cited by 12 · see citing works

Solving Proof Block Problems Using Large Language Models

Poulsen, S., Sarsa, S., Prather, J. E., Leinonen, J., Becker, B. A., Hellas, A., Denny, P. C., & Reeves, B. N. (2024) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3626252.3630928 · Cited by 12 · see citing works

ChatGPT Incorrectness Detection in Software Reviews

Tanzil, M. H., Khan, J. Y., & Uddin, G. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639194 · Cited by 12 · see citing works

Reliability Check: An Analysis of GPT-3’s Response to Sensitive Topics and Prompt Wording

Khatun, A., & Brown, D. G. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.8 · Cited by 12 · see citing works

P-Adapters: Robustly Extracting Factual Information from Language Models with Diverse PromptsPreprint

Newman, B., Choubey, P. K., & Rajani, N. F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.07280 · Cited by 12 · see citing works

The Factuality of Large Language Models in the Legal Domain

Hamdani, R. E., Bonald, T., Malliaros, F. D., Holzenberger, N., & Suchanek, F. M. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679961 · Cited by 11 · see citing works

Improving Factual Consistency for Knowledge-Grounded Dialogue Systems via Knowledge Enhancement and Alignment

Xue, B., Wang, W., Wang, H., Mi, F., Wang, R., Wang, Y., Shang, L., Jiang, X., Liu, Q., & Wong, K. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.525 · Cited by 11 · see citing works

From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency

Ohmer, X. I., Bruni, E., & Hupkes, D. (2024) · Computational Linguistics · doi:10.1162/coli_a_00529 · Cited by 11 · see citing works

Neural Knowledge Bank for Pretrained Transformers

Dai, D., Jiang, W., Dong, Q., Lyu, Y., & Sui, Z. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44696-2_60 · Cited by 11 · see citing works

DuCL: Dual-stage contrastive learning framework for Chinese semantic textual matching

Zuo, Y., Lü, W., Peng, X., Wang, S., Zhang, W., & Qiao, X. (2023) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2022.108574 · Cited by 10 · see citing works

Emergent Modularity in Pre-trained Transformers

Zhang, Z., Zeng, Z., Lin, Y., Xiao, C., Wang, X., Han, X., Liu, Z., Xie, R., Sun, M., & Zhou, J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.250 · Cited by 10 · see citing works

Language Models with Rationality

Kassner, N., Tafjord, O., Sabharwal, A., Richardson, K., Schuetze, H., & Clark, P. A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.877 · Cited by 10 · see citing works

Back to Square One: Bias Detection, Training and Commonsense Disentanglement in the Winograd SchemaPreprint

Elazar, Y., Zhang, H., Goldberg, Y., & Roth, D. (2021) · arXiv (Cornell University) · Cited by 10 · see citing works

A decision support system in precision medicine: contrastive multimodal learning for patient stratification

Yin, Q., Zhong, L. L. D., Song, Y., Bai, L., Wang, Z., Chen, L., Xu, R. Y. D., & Yang, X. (2023) · Annals of Operations Research · doi:10.1007/s10479-023-05545-6 · Cited by 9 · see citing works

Metamorphic Testing of Large Language Models for Natural Language Processing

Cho, S. B., Ruberto, S., & Terragni, V. (2025) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme64153.2025.00025 · Cited by 9 · see citing works

Accurate, yet inconsistent? Consistency Analysis on Language Understanding ModelsPreprint

Jang, M., Kwon, D. S., & Lukasiewicz, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.06665 · Cited by 9 · see citing works

Improving Language Models’ Meaning Understanding and Consistency by Learning Conceptual Roles from Dictionary

Jang, M., & Lukasiewicz, T. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.527 · Cited by 4 · see citing works

Consistency and completeness of knowledge acquired by language models

Kassner, N. (2025) · Electronic Theses of LMU Munich (Ludwig-Maximilians-Universität München) · doi:10.5282/edoc.36004

How Often Are Errors in Natural Language Reasoning Due to Paraphrastic Variability?

Srikanth, N. P., Carpuat, M. J., & Rudinger, R. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00692 · Cited by 5 · see citing works

Measuring Reliability of Large Language Models through Semantic ConsistencyPreprint

Raj, H., Rosati, D., & Majumdar, S. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.05853 · Cited by 9 · see citing works