How much do language models copy from their training data? Evaluating linguistic novelty in text generation using RAVEN

Attention Is All You Need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works

Long Short-Term Memory

Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works

Deep learning

LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works

Language Models are Few-Shot Learners

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works

Advances in Neural Information Processing Systems 19

(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,037 · see citing works

Advances in Neural Information Processing Systems 29

Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881 · see citing works

On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜

Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · see citing works

Proceedings of the 25th international conference on Machine learning

Cohen, W. W., McCallum, A., & Roweis, S. T. (2008) · Cited by 5,691 · see citing works

Pointer Sentinel Mixture ModelsPreprint

Merity, S., Xiong, C., Bradbury, J., & Socher, R. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1609.07843 · Cited by 5,051 · see citing works

Membership Inference Attacks Against Machine Learning Models

Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · see citing works

Moses: open source toolkit for statistical machine translation

Koehn, P., Zens, R., Dyer, C., Bojar, O., Constantin, A., Herbst, E., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., et al. (2007) · Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions - ACL '07 · doi:10.3115/1557769.1557821 · Cited by 4,735 · cited 1× · see citing works

Connectionism and cognitive architecture: A critical analysis

Fodor, J. A., & Pylyshyn, Z. W. (1988) · Cognition · doi:10.1016/0010-0277(88)90031-5 · Cited by 4,022 · see citing works

fairseq: A Fast, Extensible Toolkit for Sequence Modeling

Ott, M., Edunov, S., Baevski, A., Fan, A., Gross, S., Ng, N., Grangier, D., & Auli, M. (2019) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n19-4009 · Cited by 3,538 · see citing works

Transformer-XL: Attentive language models beyond a fixed-length context

Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · see citing works

BERTScore: Evaluating Text Generation with BERT

Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865 · see citing works

ELIZA — a computer program for the study of natural language communication between man and machine

Weizenbaum, J. (1983) · Communications of the ACM · doi:10.1145/357980.357991 · Cited by 2,765 · see citing works

The Child's Learning of English Morphology

Berko, J. (1958) · WORD · doi:10.1080/00437956.1958.11659661 · Cited by 2,504 · see citing works

The Cambridge Grammar of the English Language

Huddleston, R., & Pullum, G. (2005) · Zeitschrift für Anglistik und Amerikanistik · doi:10.1515/zaa-2005-0209 · Cited by 2,410 · see citing works

Understanding deep learning (still) requires rethinking generalization

Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2021) · Communications of the ACM · doi:10.1145/3446776 · Cited by 2,400 · see citing works

Scaling Laws for Neural Language Models

Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · Cited by 1,749 · see citing works

Moses : open source toolkit for statistical machine translation

Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., et al. (2007) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 1,547 · see citing works

Evaluating Large Language Models Trained on Code

Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436 · see citing works

The Curious Case of Neural Text Degeneration

Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304 · see citing works

The 26th USENIX Security Symposium 2017

Αλεξόπουλος, Ν., Kiayias, A., Zacharias, T., & Talviste, R. (2017) · USENIX Security Symposium · Cited by 1,168 · see citing works

Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data

Bender, E. M., & Koller, A. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.463 · Cited by 1,082 · cited 1× · see citing works

Overregularization in language acquisition

Marcus, G. F., Pinker, S., Ullman, M., Hollander, M., Rosen, T. J., Xu, F., & Clahsen, H. (1992) · Monographs of the Society for Research in Child Development · doi:10.2307/1166115 · Cited by 1,065 · see citing works

Neural Machine Translation of Rare Words with Subword Units

Sennrich, R., Haddow, B., & Birch, A. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1162 · Cited by 974 · see citing works

Rules vs. analogy in English past tenses: a computational/experimental study

Albright, A., & Hayes, B. (2003) · Cognition · doi:10.1016/s0010-0277(03)00146-x · Cited by 777 · see citing works

Overfitting and undercomputing in machine learning

Dietterich, T. (1995) · ACM Computing Surveys · doi:10.1145/212094.212114 · Cited by 776 · see citing works

Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation

Gatt, A., & Krahmer, E. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.5477 · Cited by 776 · see citing works

CTRL: A Conditional Transformer Language Model for Controllable Generation

Keskar, N. S., McCann, B., Varshney, L. R., Xiong, C., & Socher, R. (2019) · arXiv · doi:10.48550/arxiv.1909.05858 · Cited by 751 · see citing works

Extracting Training Data from Large Language Models

Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07805 · Cited by 727 · see citing works

Universal grammar

Montague, R. (1970) · Theoria · doi:10.1111/j.1755-2567.1970.tb00434.x · Cited by 703 · see citing works

Evaluating the Factual Consistency of Abstractive Text Summarization

Kryscinski, W., McCann, B., Xiong, C., & Socher, R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.750 · Cited by 669 · see citing works

Using the Output Embedding to Improve Language Models

Press, O., & Wolf, L. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-2025 · Cited by 642 · see citing works

Proceedings of the 36th annual meeting on Association for Computational Linguistics -

(1998) · doi:10.3115/980845 · Cited by 636 · see citing works

The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks

Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., & Song, D. (2018) · arXiv · doi:10.48550/arxiv.1802.08232 · Cited by 629 · see citing works

Generalisation of regular and irregular morphological patterns

Prasada, S., & Pinker, S. A. (1993) · Language and Cognitive Processes · doi:10.1080/01690969308406948 · Cited by 547 · see citing works

Fluid Concepts and Creative Analogies: Computer Models of the Fundamental Mechanisms of Thought

Hofstadter, D. R. (1995) · Cited by 527 · see citing works

Recurrent Neural Network Grammars

Dyer, C., Kuncoro, A., Ballesteros, M., & Smith, N. A. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1024 · Cited by 517 · see citing works

Colorless Green Recurrent Networks Dream Hierarchically

Gulordava, K., Bojanowski, P., Grave, É., Linzen, T., & Baroni, M. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1108 · Cited by 489 · see citing works

Constituency Parsing with a Self-Attentive Encoder

Kitaev, N., & Klein, D. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1249 · Cited by 483 · see citing works

Untitled work

(2020) · arXiv (Cornell University) · Cited by 461 · see citing works

Texygen: A Benchmarking Platform for Text Generation Models

Zhu, Y., Lu, S., Zheng, L., Guo, J., Zhang, W., Wang, J., & Yu, Y. (2018) · The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval · doi:10.1145/3209978.3210080 · Cited by 408 · see citing works

The Price of Linguistic Productivity

Yang, C. D. (2016) · The MIT Press eBooks · doi:10.7551/mitpress/9780262035323.001.0001 · Cited by 340 · see citing works

The acquisition of regular and irregular past tense forms

Kuczaj, S. A. (1977) · Journal of Verbal Learning and Verbal Behavior · doi:10.1016/s0022-5371(77)80021-2 · Cited by 323 · see citing works

Improving Neural Language Models with a Continuous CachePreprint

Grave, E., Joulin, A., & Usunier, N. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1612.04426 · Cited by 316 · see citing works

Deduplicating Training Data Makes Language Models Better

Lee, K., Ippolito, D., Nystrom, A., Zhang, C., Eck, D., Callison-Burch, C., & Carlini, N. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.577 · Cited by 304 · see citing works

The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence

Marcus, G. (2020) · arXiv · doi:10.48550/arxiv.2002.06177 · Cited by 297 · see citing works

The Price of Linguistic Productivity: How Children Learn to Break the Rules of Language

Yang, C. D. (2016) · Project Muse (Johns Hopkins University) · Cited by 296 · see citing works

Generalization through Memorization: Nearest Neighbor Language Models

Khandelwal, U., Levy, O., Jurafsky, D., Zettlemoyer, L., & Lewis, M. (2019) · arXiv · doi:10.48550/arxiv.1911.00172 · Cited by 285 · see citing works

Lexical word-formation

Bauer, L., & Huddleston, R. (2002) · Cambridge University Press eBooks · doi:10.1017/9781316423530.020 · Cited by 267 · see citing works

Adaptive Input Representations for Neural Language ModelingPreprint

Baevski, A., & Auli, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.10853 · Cited by 255 · see citing works

Dialogue Natural Language Inference

Welleck, S., Weston, J., Szlam, A., & Cho, K. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1363 · Cited by 249 · see citing works

Productivity and Reuse in Language

O’Donnell, T. J. (2015) · The MIT Press eBooks · doi:10.7551/mitpress/9780262028844.001.0001 · Cited by 220 · see citing works

Slot and frame patterns and the development of the determiner category

Pine, J., & Lieven, E. (1997) · Applied Psycholinguistics · doi:10.1017/s0142716400009930 · Cited by 219 · see citing works

Automatic evaluation of text coherence: models and representations

Lapata, M., & Barzilay, R. (2005) · Cited by 218 · see citing works

Measuring Compositional Generalization: A Comprehensive Method on Realistic DataPreprint

Keysers, D., Schärli, N., Scales, N., Buisman, H., Furrer, D., Kashubin, S., Momchev, N., Sinopalnikov, D., Stafiniak, Ł., Tihon, T., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.09713 · Cited by 213 · see citing works

Evaluation of Text Generation: A Survey

Celikyilmaz, A., Clark, E., & Gao, J. (2020) · arXiv · doi:10.48550/arxiv.2006.14799 · Cited by 207 · see citing works

Quantifying Memorization Across Neural Language Models

Carlini, N., Ippolito, D., Jagielski, M., Lee, K., Tramer, F., & Zhang, C. (2022) · arXiv · doi:10.48550/arxiv.2202.07646 · Cited by 204 · see citing works

Compositionality Decomposed: How do Neural Networks Generalise?

Hupkes, D., Dankers, V., Mul, M., & Bruni, E. (2020) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11674 · Cited by 194 · see citing works

Unifying Human and Statistical Evaluation for Natural Language Generation

Hashimoto, T. B., Zhang, H., & Liang, P. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1169 · Cited by 161 · see citing works

COGS: A Compositional Generalization Challenge Based on Semantic Interpretation

Kim, N., & Linzen, T. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.731 · Cited by 153 · see citing works

Ontogeny and phylogeny of language

Yang, C. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1216803110 · Cited by 142 · see citing works

Do Massively Pretrained Language Models Make Better Storytellers?

See, A., Pappu, A., Saxena, R., Yerukola, A., & Manning, C. D. (2019) · Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL) · doi:10.18653/v1/k19-1079 · Cited by 139 · see citing works

A Systematic Assessment of Syntactic Generalization in Neural Language Models

Hu, J., Gauthier, J., Qian, P., Wilcox, E., & Levy, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.158 · Cited by 134 · see citing works

Systematicity in Connectionist Language Learning

Hadley, R. F. (1994) · Mind & Language · doi:10.1111/j.1468-0017.1994.tb00225.x · Cited by 132 · see citing works

PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking

Rashkin, H., Çelikyılmaz, A., Choi, Y., & Gao, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.349 · Cited by 130 · see citing works

Don’t Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Training

Li, M., Roller, S., Kulikov, I., Welleck, S., Boureau, Y.-L., Cho, K., & Weston, J. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.428 · Cited by 128 · see citing works

Is GPT-3 text indistinguishable from human text? Scarecrow: A framework for scrutinizing machine text

Dou, Y., Forbes, M., Koncel-Kedziorski, R., Smith, N., & Choi, Y. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.501 · Cited by 118 · see citing works

Productivity and Reuse in Language: A Theory of Linguistic Computation and Storage

O’Donnell, T. J. (2015) · Project Muse (Johns Hopkins University) · Cited by 100 · see citing works

Efficient softmax approximation for GPUs

Grave, É., Joulin, A., Cissé, M., & Jeǵou, H. (2017) · International Conference on Machine Learning · Cited by 96 · see citing works

Efficient Second-Order TreeCRF for Neural Dependency Parsing

Zhang, Y., Li, Z., & Zhang, M. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.302 · Cited by 89 · see citing works

Still not systematic after all these years: On the compositional skills of sequence-to-sequence recurrent networksPreprint

Lake, B., & Baroni, M. (2017) · International Conference on Learning Representations · doi:10.48550/arxiv.1711.00350 · Cited by 85 · see citing works

When Do You Need Billions of Words of Pretraining Data?

Zhang, Y., Warstadt, A., Li, X., & Bowman, S. R. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.90 · Cited by 85 · see citing works

Language Model Evaluation Beyond Perplexity

Meister, C., & Cotterell, R. (2021) · doi:10.18653/v1/2021.acl-long.414 · Cited by 74 · see citing works

Does Syntax Need to Grow on Trees? Sources of Hierarchical Inductive Bias in Sequence-to-Sequence Networks

McCoy, R. T., Frank, R., & Linzen, T. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00304 · Cited by 72 · see citing works

Deduplicating Training Data Mitigates Privacy Risks in Language Models

Kandpal, N., Wallace, E., & Raffel, C. (2022) · arXiv · doi:10.48550/arxiv.2202.06539 · Cited by 68 · see citing works

GLEU: Automatic Evaluation of Sentence-Level Fluency

Mutton, A., Dras, M., Wan, S., & Dale, R. (2007) · Cited by 65 · see citing works

On Compositional Generalization of Neural Machine Translation

Li, Y., Yin, Y.-J., Chen, Y., & Zhang, Y. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.368 · Cited by 60 · see citing works

Do Language Models Plagiarize?

Lee, J., Le, T., Chen, J., & Lee, D. (2023) · Proceedings of the ACM Web Conference 2023 · doi:10.1145/3543507.3583199 · Cited by 51 · see citing works

Enhancing the Transformer with Explicit Relational Encoding for Math Problem SolvingPreprint

Schlag, I., Smolensky, P., Fernandez, R., Jojić, N., Schmidhuber, J., & Gao, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.06611 · Cited by 45 · see citing works

Frequency Effects on Syntactic Rule Learning in Transformers

Wei, J., Garrette, D., Linzen, T., & Pavlick, E. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.72 · Cited by 45 · see citing works

Towards Tracing Knowledge in Language Models Back to the Training Data

Akyürek, E., Bolukbasi, T., Liu, F., Xiong, B., Tenney, I., Andreas, J., & Guu, K. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.180 · Cited by 40 · see citing works

Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languages

Ravfogel, S., Goldberg, Y., & Linzen, T. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1356 · Cited by 39 · see citing works

Inflectional morphology and related matters

Palmer, F. R., Huddleston, R., & Pullum, G. K. (2002) · Cambridge University Press eBooks · doi:10.1017/9781316423530.019 · Cited by 38 · see citing works

Scarecrow: A Framework for Scrutinizing Machine Text.Preprint

Dou, Y., Forbes, M., Koncel-Kedziorski, R., Smith, N. A., & Choi, Y. (2021) · arXiv (Cornell University) · Cited by 17 · see citing works

Measuring Causal Effects of Data Statistics on Language Model's `Factual' PredictionsPreprint

Elazar, Y., Kassner, N., Ravfogel, S., Feder, A., Ravichander, A., Mosbach, M., Belinkov, Y., Schütze, H., & Goldberg, Y. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.14251 · Cited by 15 · see citing works

Recurrent babbling: evaluating the acquisition of grammar from limited input data

Pannitto, L., & Herbelot, A. (2020) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2020.conll-1.13 · Cited by 12 · see citing works

This is a BERT. Now there are several of them. Can they generalize to novel words?

Haley, C. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.31 · Cited by 11 · see citing works

ORCA: Interpreting Prompted Language Models via Locating Supporting Data Evidence in the Ocean of Pretraining DataPreprint

Han, X., & Tsvetkov, Y. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.12600 · Cited by 10 · see citing works

Probing BERT’s priors with serial reproduction chains

Yamakoshi, T., Griffiths, T. L., & Hawkins, R. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.314 · Cited by 8 · see citing works

Distinguishing rule- and exemplar-based generalization in learning systemsPreprint

Dasgupta, I., Grant, E., & Griffiths, T. L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.04328 · Cited by 7 · see citing works

Evaluating Distributional Distortion in Neural Language ModelingPreprint

LeBrun, B., Sordoni, A., & O’Donnell, T. J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2203.12788 · Cited by 7 · see citing works

OPT: Open Pre-trained Transformer Language Models

Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.01068

The rise and potential of large language model based agents: a survey

Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970 · see citing works

Dissociating language and thought in large language models

Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.011 · Cited by 345 · see citing works

The Rise and Potential of Large Language Model Based Agents: A Survey

Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972 · see citing works

Delving into LLM-assisted writing in biomedical publications through excess vocabulary

Kobak, D., González-Márquez, R., Horvát, E.-Á., & Lause, J. (2025) · Science Advances · doi:10.1126/sciadv.adt3813 · Cited by 213 · see citing works

Analyzing Leakage of Personally Identifiable Information in Language Models

Lukas, N., Salem, A., Sim, R., Tople, S., Wutschitz, L., & Zanella-Béguelin, S. (2023) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46215.2023.10179300 · Cited by 181 · see citing works

Theory Is All You Need: AI, Human Cognition, and Causal Reasoning

(2024) · doi:10.1287/stsc.2024.0189 · Cited by 151 · see citing works

On the creativity of large language models

Franceschelli, G., & Musolesi, M. (2025) · AI & SOCIETY · doi:10.1007/s00146-024-02127-3 · Cited by 113 · see citing works

Language Model Behavior: A Comprehensive Survey

Chang, T. A., & Bergen, B. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00492 · Cited by 105 · see citing works

Symbols and grounding in large language models

Pavlick, E. (2023) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2022.0041 · Cited by 95 · see citing works

Linguistically inspired roadmap for building biologically reliable protein language models

Vu, M. H., Akbar, R., Robert, P. A., Świątczak, B., Sandve, G. K., Greiff, V., & Haug, D. T. T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00637-1 · Cited by 57 · see citing works

TemporalWiki: A Lifelong Benchmark for Training and Evaluating Ever-Evolving Language Models

Jang, J., Ye, S., Lee, C.-H., Yang, S., Shin, J., Han, J., Kim, G., & Seo, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.418 · Cited by 60 · see citing works

The Ethical Implications of Generative Audio Models: A Systematic Literature Review

Barnett, J. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604686 · Cited by 47 · see citing works

The homogenizing effect of large language models on human expression and thought

Sourati, Z., Ziabari, A. S., & Dehghani, M. (2026) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2026.01.003 · Cited by 62 · see citing works

See Widely, Think Wisely: Toward Designing a Generative Multi-agent System to Burst Filter Bubbles

Zhang, Y., Sun, J., Feng, L., Yao, C., Fan, M., Zhang, L., Wang, Q., Geng, X., & Rui, Y. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642545 · Cited by 44 · see citing works

The Next Chapter: A Study of Large Language Models in Storytelling

Xie, Z., Cohn, T., & Lau, J. H. (2023) · International Conference on Natural Language Generation · doi:10.18653/v1/2023.inlg-main.23 · Cited by 41 · see citing works

The structure and statistics of language jointly shape cross-frequency neural dynamics during spoken language comprehension

Weissbart, H., & Martin, A. E. (2024) · Nature Communications · doi:10.1038/s41467-024-53128-1 · Cited by 33 · see citing works

Deconstructing the ethics of large language models from long-standing issues to new-emerging dilemmas: a survey

Deng, C., Duan, Y., Jin, X., Chang, H., Tian, Y., Liu, H., Wang, Y., Gao, K., Zou, H. P., Jin, Y., et al. (2025) · AI and Ethics · doi:10.1007/s43681-025-00797-3 · Cited by 49 · see citing works

Staying ahead with generative artificial intelligence for learning: navigating challenges and opportunities with 5Ts and 3Rs

Lee, A. V. Y. (2024) · Asia Pacific Journal of Education · doi:10.1080/02188791.2024.2305171 · Cited by 20 · see citing works

Training Data Extraction From Pre-trained Language Models: A Survey

Ishihara, S. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.23 · Cited by 26 · see citing works

Usage-based constructionist approaches and large language models

Goldberg, A. E. (2024) · Constructions and Frames · doi:10.1075/cf.23017.gol · Cited by 21 · see citing works

LMs go Phishing: Adapting Pre-trained Language Models to Detect Phishing Emails

Misra, K., & Rayz, J. T. (2022) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat55865.2022.00028 · Cited by 16 · see citing works

Findings of the The RuATD Shared Task 2022 on Artificial Text Detection in Russian

Shamardina, T., Mikhailov, V., Chernianskii, D., Fenogenova, A., Saidov, M., Valeeva, A., Shavrina, T., Smurov, I., Tutubalina, E. V., & Artemova, E. (2022) · Computational Linguistics and Intellectual Technologies · doi:10.28995/2075-7182-2022-21-497-511 · Cited by 22 · see citing works

Impact of Co-occurrence on Factual Knowledge of Large Language Models

Kang, C., & Choi, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.518 · Cited by 18 · see citing works

Transformer-based large language foundation models for text generation: A comprehensive literature review for different languages and application domains

de Oliveira, R. S., & Nascimento, E. G. S. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104477 · Cited by 12 · see citing works

A multidimensional study of AI adoption among University students in teacher education programs

Bećirović, S., Polz, E., & Tinkel, I. (2025) · Smart Learning Environments · doi:10.1186/s40561-025-00422-0 · Cited by 14 · see citing works

Are Language Models More Like Libraries or Like Librarians? Bibliotechnism, the Novel Reference Problem, and the Attitudes of LLMs

Lederman, H., & Mahowald, K. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00690 · Cited by 12 · see citing works

You can’t pick your neighbors, or can you? When and How to Rely on Retrieval in the kNN-LM

Drozdov, A., Wang, S., Rahimi, R., McCallum, A., Zamani, H., & Iyyer, M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.218 · Cited by 10 · see citing works

Language-Models-as-a-Service: Overview of a New Paradigm and its Challenges

La Malfa, E., Petrov, A., Frieder, S., Weinhuber, C., Burnell, R., Nazar, R., Cohn, A., Shadbolt, N., & Wooldridge, M. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15865 · Cited by 9 · see citing works

DP-TabICL: In-Context Learning with Differentially Private Tabular Data

Carey, A. N., Bhaila, K., Edemacu, K., & Wu, X. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10826053 · Cited by 8 · see citing works

Echoes in AI: Quantifying lack of plot diversity in LLM outputs

Xu, W., Jojić, N., Rao, S., Brockett, C., & Dolan, B. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2504966122 · Cited by 14 · see citing works

Beyond Individual Concerns: Multi-user Privacy in Large Language Models

Zhan, X., Seymour, W., & Such, J. M. (2024) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3640794.3665883 · Cited by 7 · see citing works

Derivational morphology reveals analogical generalization in large language models

Hofmann, V., Weissweiler, L., Mortensen, D. R., Schütze, H., & Pierrehumbert, J. B. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2423232122 · Cited by 13 · see citing works

High-Fidelity Cellular Network Control-Plane Traffic Generation without Domain Knowledge

Kong, Z. J., Hu, N., Hu, Y., Meng, J., & Koral, Y. (2024) · ACM Internet Measurement Conference (IMC) · doi:10.1145/3646547.3688422 · Cited by 5 · see citing works

kNN-LM Does Not Improve Open-ended Text Generation

Wang, S., Song, Y., Drozdov, A., Garimella, A., Manjunatha, V., & Iyyer, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.929 · Cited by 6 · see citing works

LongStory: Coherent, Complete and Length Controlled Long Story Generation

Park, K., Yang, N., & Jung, K. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-2253-2_15 · Cited by 7 · see citing works

Artificial intelligence and its impact on creativity and diversity: An empirical study of large language model-generated product ideas

Terwiesch, C., Meincke, L., Girotra, K., Mollick, E., Nave, G., & Ulrich, K. T. (2026) · Production and Operations Management · doi:10.1177/10591478261474243 · Cited by 191 · see citing works

An Accidental Benchmark: The History, Contingent Power, and Lasting Traces of the GTZAN Dataset

Jerzak, A. (2025) · Digital Society · doi:10.1007/s44206-025-00191-w · Cited by 3 · see citing works

Self-repetition in abstractive neural summarizers

Salkar, N. (2022) · doi:10.17760/d20467253 · Cited by 3 · see citing works

Psittacines of Innovation? Assessing the True Novelty of AI CreationsPreprint

Mukherjee, A. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4764101 · Cited by 3 · see citing works

Generative Models as a Complex Systems Science: How Can We Make Sense of Large Language Model Behavior?

Holtzman, A., West, P., & Zettlemoyer, L. (2025) · Journal of Social Computing · doi:10.23919/jsc.2025.0009 · Cited by 14 · see citing works

A Semantics-Aware Head-Driven Approach for Multilingual Dependency Parsing

Nguyễn, T. M. H., & Le-Hong, P. (2025) · IEEE Access · doi:10.1109/access.2025.3601225 · Cited by 2 · see citing works

Detecting Training Data For Large Language Models: A Survey

Yang, C., Li, J., Shulin, L. A. N., Wang, Y., Du, H., Gong, C., Yao, X., Niyato, D. T., & Zhu, L. (2026) · ACM Computing Surveys · doi:10.1145/3779430 · Cited by 2 · see citing works

Fine-Tuning GPT-3 for Synthetic Danish News Generation

Almasi, M., & Schiønning, A. (2023) · International Conference on Natural Language Generation · doi:10.18653/v1/2023.inlg-main.4 · Cited by 2 · see citing works

Modelling child comprehension: A case of suffixal passive construction in Korean

Shin, G., & Mun, S. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101701 · Cited by 2 · see citing works

ChatGPT vs Bard: Which is a Better Writer?

Ng, A. L., & Ong, J. (2024) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai59869.2024.00066 · Cited by 3 · see citing works

Snoopy: An Online Interface for Exploring the Effect of Pretraining Term Frequencies on Few-Shot LM Performance

Razeghi, Y., Mekala, R. R., Logan, R. L., Gardner, M., & Singh, S. (2022) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2022.emnlp-demos.39 · Cited by 2 · see citing works

On using distribution-based compositionality assessment to evaluate compositional generalisation in machine translation

Moisio, A., Creutz, M., & Kurimo, M. (2023) · GenBench Workshop on Generalisation (Benchmarking) in NLP (GenBench) · doi:10.18653/v1/2023.genbench-1.17 · Cited by 1 · see citing works

An Efficient Approach for Enhancing GenAI Trustworthiness

Yehezkel, A. (2024) · International Conference on Machine Learning and Applications · doi:10.1109/icmla61862.2024.00292 · Cited by 1 · see citing works

“It is important to consult” a linguist: Verb-Argument Constructions in ChatGPT and human experts’ medical and financial advice

Casal, J. E., Stewart, C. M., & Windsor, A. J. (2025) · PLoS ONE · doi:10.1371/journal.pone.0324611 · Cited by 2 · see citing works

The compositionality continuum as a principle for studying the neural basis of intelligence

Riveland, R., Pouget, A., & Driscoll, L. N. (2026) · Nature Neuroscience · doi:10.1038/s41593-026-02382-1 · Cited by 2 · see citing works

Language Models Understand Us, Poorly

Moore, J. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.16 · Cited by 2 · see citing works

Text Linguistics and the Use of Linguistic Data in Modern Technologies: Prospects for Development

Aliyeva, G. B. (2023) · Futurity of Social Sciences. · doi:10.57125/fs.2023.06.20.02 · Cited by 5 · see citing works

Creative minds like ours? Large Language Models and the creative aspect of language use

Carchidi, V. J. (2024) · Biolinguistics · doi:10.5964/bioling.13507 · Cited by 5 · see citing works

Incremental Processing of Principle B: Mismatches Between Neural Models and Humans

Davis, F. (2022) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2022.conll-1.11 · Cited by 1 · see citing works

The Structure and Statistics of Language Jointly Shape Cross-frequency Neural Dynamics During Spoken Language ComprehensionPreprint

Weissbart, H., & Martin, A. E. (2023) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2023.10.06.561087 · Cited by 1 · see citing works

LANG: A Lesson Plan Generation Framework via Multi-Form Interaction with Large Language ModelsPreprint

Ouyang, Y., Quan, J., Wang, H., YaWen, Z., & Chen, L. (2025) · Research Square · doi:10.21203/rs.3.rs-6808103/v1 · Cited by 1 · see citing works

Assessing the influence of attractor-verb distance on grammatical agreement in humans and language models

Zacharopoulos, C.-N., Desbordes, T., & Sablé-Meyer, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.998

EpiK-Eval: Evaluation for Language Models as Epistemic Models

Prato, G., Huang, J., Parthasarathi, P., Sodhani, S., & Chandar, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.593 · Cited by 1 · see citing works

Charting the Parrot's Song: A Maximum Mean Discrepancy Approach to Measuring AI Novelty, Originality, and DistinctivenessPreprint

Mukherjee, A., & Chang, H. H. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5213450

A plug-and-play method for linguistic alignment in language models

Pang, K., Bai, M., Yang, J., Gao, Y., Jiang, M., & Huang, Y. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113597

Do Language Models Know Who Did What to Whom?

Denning, J. M., Guo, X., Snefjella, B., & Blank, I. (2026) · Open Mind · doi:10.1162/opmi.a.365 · Cited by 2 · see citing works

FEMA: Emotion-Driven Personification in Generative Agents

Shao, S., Qin, X., Wu, S., Lin, G., Zhang, Y., & Chen, L. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11229401 · Cited by 1 · see citing works

Automatic generation of ESL learning materials based on CEFR levels using reinforcement-tuned LLMs

Zuo, Y. (2025) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00762-3

A Cross-Cultural Multilingual Text-to-Text Generation Framework Using Novel SanskritiGenAI Model for Preserving and Digitizing Cultural Heritage

Murthy, J. S., Dhanashekar, K., & Siddesh, G. M. (2026) · Lecture notes in networks and systems · doi:10.1007/978-981-96-8799-2_20

Adaptive prototype and relation-aware distillation for multimodal sentiment analysis with incomplete data

Sun, X., Luo, L., & Peng, H. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.133397

A Curriculum-Based RAG Learning Chatbot with Dual Response Strategies: Evidence-Based Explanations and Question-Prompting Responses

Ha, O.-S. (2026) · Journal of Digital Contents Society · doi:10.9728/dcs.2026.27.3.821

Data-Based Natural Language Generation : Evaluation and Explainability

Faille, J. (2023) · theses.fr (ABES) · doi:10.70675/3e73dd43z7aa6z4405za5cez27c78b408a3c

AI and the Reduction of Truth: A Eucharistic Alternative

Reilly, C. M. (2026) · Religions · doi:10.3390/rel17050551

Enhancing dependency parsing with graph neural network-learned embeddings from universal dependency treebanks

Ngo, T. Q., Nguyễn, T. M. H., & Le-Hong, P. (2026) · Computer Speech & Language · doi:10.1016/j.csl.2026.102000

Large language models illuminate the mechanistic underpinnings of the creative aspect of language use (CALU), long regarded as a mystery

Sripada, C., McInnerney, A., & Lewis, R. L. (2026) · Behavioral and Brain Sciences · doi:10.1017/s0140525x26104725

Evaluation of Large Language Models as Tools, Models, and Partners in Creative Thinking Research: A Selective Narrative Review with the GCA Framework

Huang, K., Liu, C., & Yang, J. (2026) · Journal of Intelligence · doi:10.3390/jintelligence14090218

Can DeepSeek replace humans in creating options: A comparison on the linguistic features of human options and AI-generated options

Chen, Z. (2026) · SHS Web of Conferences · doi:10.1051/shsconf/202624002001

Evaluating n-Gram Novelty of Language Models Using Rusty-DAWG

Merrill, W., Smith, N. A., & Elazar, Y. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.800 · Cited by 8 · see citing works

RUSSIAN NATURAL LANGUAGE GENERATION: CREATION OF A LANGUAGE MODELING DATASET AND EVALUATION WITH MODERN NEURAL ARCHITECTURES

Shaheen, Z., Wohlgenannt, G., Zaity, B., Mouromtsev, D. I., & Pak, V. (2020) · Computational Linguistics and Intellectual Technologies · doi:10.28995/2075-7182-2020-19-644-657 · Cited by 1 · see citing works

CopyBench: Measuring Literal and Non-Literal Reproduction of Copyright-Protected Text in Language Model Generation

Chen, T., Asai, A., Mireshghallah, N., Min, S., Grimmelmann, J., Choi, Y., Hajishirzi, H., Zettlemoyer, L., & Koh, P. W. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.844 · Cited by 10 · see citing works

Russian Natural Language Generation: Creation of a Language Modelling Dataset and Evaluation with Modern Neural ArchitecturesPreprint

Shaheen, Z., Wohlgenannt, G., Zaity, B., Mouromtsev, D. I., & Pak, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.02470 · Cited by 1 · see citing works

LLM generation novelty through the lens of semantic similarity

Davydov, P., Prabhu, A., Bethge, M., Nguyen, E., & Oh, S. J. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.1942