Gradient Ascent Post-training Enhances Language Model Generalization

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works

Language Models are Few-Shot Learners

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works

PaLM: Scaling Language Modeling with Pathways

Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · see citing works

MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms

Amini, A., Gabriel, S., Lin, S., Koncel-Kedziorski, R., Choi, Y., & Hajishirzi, H. (2019) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n19-1245 · Cited by 956 · see citing works

Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset

Rashkin, H., Smith, E. M., Li, M., & Boureau, Y.-L. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1534 · Cited by 883 · see citing works

Averaging Weights Leads to Wider Optima and Better GeneralizationPreprint

Izmailov, P., Podoprikhin, D. A., Garipov, T., Vetrov, D., & Wilson, A. G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.05407 · Cited by 818 · see citing works

Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?

Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.759 · Cited by 755 · see citing works

Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?

Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022) · arXiv · doi:10.48550/arxiv.2202.12837 · Cited by 752 · see citing works

PubMedQA: A Dataset for Biomedical Research Question Answering

Jin, Q., Dhingra, B., Liu, Z., Cohen, W., & Lu, X. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1259 · Cited by 685 · see citing works

Multitask Prompted Training Enables Zero-Shot Task Generalization

Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.08207 · Cited by 612 · see citing works

Wizard of Wikipedia: Knowledge-Powered Conversational agentsPreprint

Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., & Weston, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01241 · Cited by 610 · see citing works

WinoGrande: An Adversarial Winograd Schema Challenge at Scale

Sakaguchi, K., Le Bras, R., Bhagavatula, C., & Choi, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6399 · Cited by 579 · see citing works

Sharpness-Aware Minimization for Efficiently Improving Generalization

Foret, P., Kleiner, A., Mobahi, H., & Neyshabur, B. (2020) · arXiv · doi:10.48550/arxiv.2010.01412 · Cited by 521 · see citing works

The Pile: An 800GB Dataset of Diverse Text for Language Modeling

Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., et al. (2021) · arXiv · doi:10.48550/arxiv.2101.00027 · Cited by 492 · see citing works

Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge

Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., & Tafjord, O. (2018) · arXiv · doi:10.48550/arxiv.1803.05457 · Cited by 462 · see citing works

GPT-NeoX-20B: An Open-Source Autoregressive Language Model

Black, S., Biderman, S., Hallahan, E., Anthony, Q. G., Gao, L., Golding, L., He, H., Leahy, C., McDonell, K., Phang, J., et al. (2022) · doi:10.18653/v1/2022.bigscience-1.9 · Cited by 407 · see citing works

Scaling Language Models: Methods, Analysis & Insights from Training Gopher

Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.11446 · Cited by 235 · see citing works

SWAD: Domain Generalization by Seeking Flat MinimaPreprint

Cha, J., Chun, S., Lee, K., Cho, H.-C., Park, S., Lee, Y., & Park, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.08604 · Cited by 209 · see citing works

Internet-Augmented Dialogue Generation

Komeili, M., Shuster, K., & Weston, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.579 · Cited by 192 · see citing works

Can You Put it All Together: Evaluating Conversational Agents’ Ability to Blend Skills

Smith, E. M., Williamson, M., Shuster, K., Weston, J., & Boureau, Y.-L. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.183 · Cited by 179 · see citing works

Knowledge Unlearning for Mitigating Privacy Risks in Language Models

Jang, J., Yoon, D., Yang, S., Cha, S., Lee, M., Logeswaran, L., & Seo, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.805 · Cited by 135 · see citing works

MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence FrontiersPreprint

Pillutla, K., Swayamdipta, S., Zellers, R., Thickstun, J., Welleck, S., Choi, Y., & Harchaoui, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.01454 · Cited by 128 · see citing works

A Contrastive Framework for Neural Text Generation

Su, Y., Lan, T., Wang, Y., Yogatama, D., Kong, L., & Collier, N. (2022) · neural information processing systems · doi:10.52202/068431-1566 · Cited by 126 · see citing works

Sharpness-Aware Minimization Improves Language Model Generalization

Bahri, D., Mobahi, H., & Tay, Y. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.508 · Cited by 50 · see citing works

PIQA: Reasoning about Physical Commonsense in Natural Language

Bisk, Y., Zellers, R., Le bras, R., Gao, J., & Choi, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6239 · Cited by 37 · see citing works

When Do Flat Minima Optimizers Work?

Kaddour, J., Liu, L., Silva, R., & Kusner, M. J. (2022) · neural information processing systems · doi:10.52202/068431-1206 · Cited by 13 · see citing works

COPA-SSE: Semi-structured Explanations for Commonsense Reasoning

Brassard, A., Heinzerling, B., Kavumba, P., & Inui, K. (2022) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4a6dosi45kib · Cited by 5 · see citing works

Baby Llama: knowledge distillation from an ensemble of teachers trained on a small dataset with no performance penalty

Timiryasov, I., & Tastet, J. (2023) · BabyLM Challenge · doi:10.18653/v1/2023.conll-babylm.24 · Cited by 38 · see citing works

Digital forgetting in large language models: a survey of unlearning methods

Blanco-Justicia, A., Jebreel, N., Manzanares-Salor, B., Sánchez, D., Domingo-Ferrer, J., Collell, G., & Eeik Tan, K. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11078-6 · Cited by 31 · see citing works

AI model disgorgement: Methods and choices

Achille, A., Kearns, M. J., Klingenberg, C., & Soatto, S. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2307304121 · Cited by 13 · see citing works

Defending against high-poisoning backdoor attacks by exploiting loss divergence

Jin, Y., Wang, R., Wang, N., He, D., Zhang, Y., & Liu, Z. (2026) · Information Fusion · doi:10.1016/j.inffus.2026.104308 · Cited by 1 · see citing works

Sharpness-Aware Minimization Improves Language Model Generalization

Bahri, D., Mobahi, H., & Tay, Y. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.508 · Cited by 50 · see citing works

GradOT: Training-free Gradient-preserving Offsite-tuning for Large Language Models

Yao, K., Tan, Z., Gao, P., Li, L., Wu, K., Wang, Y., Zhao, Y. F., Ji, Y., Zhu, J., & Wang, W. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.255

Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Models

Shi, D., Han, Z., Ostermann, S., Jin, R., Genabith, J. V., & Xiong, D. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1808

A Survey of Post-Training Scaling in Large Language Models

Lai, H., Liu, X., Gao, J., Cheng, J., Qi, Z., Xu, Y., Yao, S., Zhang, D., Du, J., Hou, Z., et al. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.140 · Cited by 8 · see citing works

G-Tuning: Improving Generalization of Pre-trained Language Models with Generative Adversarial Network

Weng, R., Cheng, W. S., & Zhang, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.291 · Cited by 2 · see citing works