On-the-Fly Attention Modulation for Neural Generation

Attention Is All You Need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works

Language Models are Few-Shot Learners

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works

BLEU: a method for automatic evaluation of machine translation

Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works

Untitled work

(2004) · Cited by 9,718 · see citing works

CIDEr: Consensus-based image description evaluation

Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · see citing works

Untitled work

(2005) · Cited by 4,152 · see citing works

Attention is not Explanation

Jain, S., & Wallace, B. C. (2019) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n19-1357 · Cited by 2,287 · see citing works

Longformer: The Long-Document Transformer

Beltagy, I., Peters, M. E., & Cohan, A. (2020) · arXiv · doi:10.48550/arxiv.2004.05150 · Cited by 2,281 · see citing works

SPICE: Semantic Propositional Image Caption Evaluation

Anderson, P., Fernando, B., Johnson, M., & Gould, S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46454-1_24 · Cited by 1,986 · see citing works

Big Bird: Transformers for Longer Sequences

Zaheer, M., Guruganesh, G., Dubey, A., Ainslie, J., Alberti, C., Ontanon, S., Pham, P., Ravula, A., Wang, Q., Yang, L., et al. (2020) · arXiv · doi:10.48550/arxiv.2007.14062 · Cited by 1,349 · see citing works

The Curious Case of Neural Text Degeneration

Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304 · see citing works

Attention is not not Explanation

Wiegreffe, S., & Pinter, Y. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1002 · Cited by 966 · see citing works

Rethinking Attention with Performers

Choromanski, K., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J., Mohiuddin, A., Kaiser, L., et al. (2020) · arXiv · doi:10.48550/arxiv.2009.14794 · Cited by 695 · see citing works

A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories

Mostafazadeh, N., Chambers, N., He, X., Parikh, D., Batra, D., Vanderwende, L., Kohli, P., & Allen, J. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/n16-1098 · Cited by 598 · see citing works

Untitled work

(2020) · arXiv (Cornell University) · Cited by 461 · see citing works

Abductive Commonsense ReasoningPreprint

Bhagavatula, C., Bras, R. J. L., Malaviya, C., Sakaguchi, K., Holtzman, A., Rashkin, H., Downey, D., Yih, S., & Choi, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.05739 · Cited by 331 · see citing works

Sharp nearby, fuzzy far away: How neural language models use context

Khandelwal, U., He, H., Qi, P., & Jurafsky, D. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1027 · Cited by 303 · see citing works

Analyzing the Structure of Attention in a Transformer Language Model

Vig, J., & Belinkov, Y. (2019) · Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w19-4808 · Cited by 301 · see citing works

An Analysis of Encoder Representations in Transformer-Based Machine Translation

Raganato, A., & Tiedemann, J. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5431 · Cited by 277 · see citing works

Neural Text Generation with Unlikelihood Training

Welleck, S., Kulikov, I., Roller, S., Dinan, E., Cho, K., & Weston, J. (2019) · arXiv · doi:10.48550/arxiv.1908.04319 · Cited by 254 · see citing works

CommonGen: A constrained text generation challenge for generative commonsense reasoning

Lin, B. Y., Zhou, W., Shen, M., Zhou, P., Bhagavatula, C., Choi, Y., & Ren, X. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.165 · Cited by 239 · cited 3× · see citing works

NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic ConstraintsPreprint

Lu, X., West, P., Zellers, R., Bras, R. L., Bhagavatula, C., & Choi, Y. (2020) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2010.12884 · Cited by 204 · see citing works

Learning to Write with Cooperative Discriminators

Holtzman, A., Buys, J., Forbes, M., Bosselut, A., Golub, D., & Choi, Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1152 · Cited by 192 · see citing works

Attention is Not Only a Weight: Analyzing Transformers with Vector Norms

Kobayashi, G., Kuribayashi, T., Yokoi, S., & Inui, K. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.574 · Cited by 178 · see citing works

Do Attention Heads in BERT Track Syntactic Dependencies?Preprint

Htut, P. M., Phang, J., Bordia, S., & Bowman, S. R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.12246 · Cited by 122 · see citing works

Neural Text Generation With Unlikelihood Training

Welleck, S., Kulikov, I., Roller, S., Dinan, E., Cho, K., & Weston, J. (2020) · International Conference on Learning Representations · Cited by 106 · see citing works

NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints

Lu, X., West, P., Zellers, R., Bras, R. J. L., Bhagavatula, C., & Choi, Y. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.339 · Cited by 79 · see citing works

Adding Interpretable Attention to Neural Translation Models Improves Word AlignmentPreprint

Zenkel, T., Wuebker, J., & DeNero, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.11359 · Cited by 75 · see citing works

Improving Neural Story Generation by Targeted Common Sense Grounding

Mao, H. H., Majumder, B. P., McAuley, J., & Cottrell, G. W. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1615 · Cited by 50 · see citing works

Dense Text-to-Image Generation with Attention Modulation

Kim, Y., Lee, J., Kim, J.-H., Ha, J.-W., & Zhu, J.-Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00708 · Cited by 115 · see citing works

ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification

Zhou, Y., Shen, T., Geng, X., Long, G., & Jiang, D. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.183 · Cited by 41 · see citing works

On the Blind Spots of Model-Based Evaluation Metrics for Text Generation

He, T., Zhang, J., Wang, T., Kumar, S., Cho, K., Glass, J. R., & Tsvetkov, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.674 · Cited by 19 · see citing works

Controlling the Focus of Pretrained Language Generation Models

Ji, J., Kim, Y., Glass, J. R., & He, T. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.260 · Cited by 4 · see citing works

Towards a Mechanistic Interpretation of Multi-Step Reasoning Capabilities of Language Models

Hou, Y., Li, J., Fei, Y., Stolfo, A., Zhou, W., Zeng, G., Bosselut, A., & Sachan, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.299 · Cited by 10 · see citing works

Decider: A Dual-System Rule-Controllable Decoding Framework for Language Generation

Xu, C., Lan, T., Ji, Y., Yu, C., Wang, W., Gao, J., Dong, Q., Qian, K., Li, P., Bi, W., et al. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3554819 · Cited by 2 · see citing works

Storycraft: Short Story Generation Using Deep Learning

Guliyani, A., Choudhary, A., Arora, L., & Munjal, G. (2025) · International Conference on Computational Intelligence and Communication Technologies (CCICT) · doi:10.1109/ccict65753.2025.00106 · Cited by 1 · see citing works

Evade the Trap of Mediocrity: Promoting Diversity and Novelty in Text Generation via Concentrating Attention

Li, W., Yi, X., Hu, J., Sun, M., & Xie, X. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.745 · Cited by 2 · see citing works

Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization

Hsieh, C.-Y., Chuang, Y.-S., Li, C.-L., Wang, Z., Le, L. T., Kumar, A., Glass, J., Ratner, A., Lee, C.-Y., Krishna, R., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.16008

Enhancing Alzheimer's Diagnosis with Pre-trained Embeddings and Attention Mechanisms

Hasan, M. S., Badhon, A. I. M., & Alam, M. G. R. (2025) · International Conference on Software Engineering and Information Management (ICSIM) · doi:10.1145/3725899.3725932

A Survey of Text Deduplication: From Syntactic Matching to Semantic Understanding

Kim, J., & Park, Y. (2026) · IEEE Access · doi:10.1109/access.2026.3658439 · Cited by 2 · see citing works

On-the-Fly Attention Modularization for Neural GenerationPreprint

Dong, Y., Bhagavatula, C., Lu, X., Hwang, J. D., Bosselut, A., Cheung, J. C. K., & Choi, Y. (2021) · arXiv (Cornell University)

Neural Data-to-Text Generation via Jointly Learning the Segmentation and Correspondence

Shen, X., Chang, E., Su, H., Niu, C., & Klakow, D. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.641 · Cited by 44 · see citing works

Advancing neural text generation

Han, S. (2021) · DR-NTU (Nanyang Technological University)

Controlling Attention Dynamics for Factual Grounding in Neural Text Generation

Rozario, R. M., Ogunbona, P., Win, K. T., & Yang, J. (2026) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn69854.2026.11729145

Neural Attention을 반영한 문장 생성 모델

이세희, & 이지형 (2017) · 한국컴퓨터정보학회 학술발표논문집