CodeBERT: A Pre-Trained Model for Programming and Natural Languages

Attention Is All You Need

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works

Adam: A Method for Stochastic Optimization

Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works

Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation

Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works

Sequence to Sequence Learning with Neural Networks

Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works

Convolutional Neural Networks for Sentence Classification

Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831 · see citing works

Deep Contextualized Word Representations

Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · see citing works

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · see citing works

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1910.13461 · Cited by 8,816 · see citing works

XLNet: Generalized Autoregressive Pretraining for Language Understanding

Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · see citing works

Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · see citing works

Moses: open source toolkit for statistical machine translation

Koehn, P., Zens, R., Dyer, C., Bojar, O., Constantin, A., Herbst, E., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., et al. (2007) · Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions - ACL '07 · doi:10.3115/1557769.1557821 · Cited by 4,735 · see citing works

Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation

Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.1078 · Cited by 3,317 · see citing works

Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

Tai, K. S., Socher, R., & Manning, C. D. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1150 · Cited by 3,081 · see citing works

A Neural Attention Model for Abstractive Sentence Summarization

Rush, A. M., Chopra, S., & Weston, J. (2015) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1044 · Cited by 2,879 · see citing works

ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks

Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109 · see citing works

How Multilingual is Multilingual BERT?

Pires, T., Schlinger, E., & Garrette, D. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1493 · Cited by 1,351 · see citing works

VideoBERT: A Joint Model for Video and Language Representation Learning

Sun, C., Myers, A., Vondrick, C., Murphy, K. P., & Schmid, C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00756 · Cited by 1,212 · see citing works

Convolutional Neural Networks for Sentence ClassificationPreprint

Kim, Y. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5882 · Cited by 1,112 · see citing works

Summarizing Source Code using a Neural Attention Model

Iyer, S. V., Konstas, I., Cheung, A., & Zettlemoyer, L. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1195 · Cited by 716 · see citing works

Deep code search

Gu, X., Zhang, H., & Kim, S. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3180155.3180167 · Cited by 539 · see citing works

CodeSearchNet Challenge: Evaluating the State of Semantic Code Search

Husain, H., Wu, H.-H., Gazit, T., Allamanis, M., & Brockschmidt, M. (2019) · arXiv · doi:10.48550/arxiv.1909.09436 · Cited by 483 · see citing works

code2seq: Generating Sequences from Structured Representations of CodePreprint

Alon, U., Brody, S., Levy, O., & Yahav, E. (2018) · arXiv (Cornell University) · doi:10.4230/oasics.slate.2021.14 · Cited by 467 · see citing works

Language Models as Knowledge Bases?Preprint

Petroni, F., Rocktäschel, T., Lewis, P., Bakhtin, A., Wu, Y., Miller, A., & Riedel, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.01066 · Cited by 364 · see citing works

An Introduction to Neural Information Retrieval

Mitra, B., & Craswell, N. (2018) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000061 · Cited by 317 · see citing works

Untitled work

(2004) · doi:10.3115/1220355.1220427 · Cited by 294 · see citing works

oLMpics-On What Language Model Pre-training Captures

Talmor, A., Elazar, Y., Goldberg, Y., & Berant, J. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00342 · Cited by 267 · see citing works

Learning and Evaluating Contextual Embedding of Source CodePreprint

Kanade, A., Maniatis, P., Balakrishnan, G., & Shi, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.00059 · Cited by 238 · see citing works

VideoBERT: A Joint Model for Video and Language Representation LearningPreprint

Sun, C., Myers, A., Vondrick, C., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.01766 · Cited by 148 · see citing works

CodeSearchNet Challenge: Evaluating the State of Semantic Code Search

Husain, H., Wang, H.-H., Gazit, T., Allamanis, M., & Brockschmidt, M. (2019) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.7908467 · Cited by 144 · see citing works

How multilingual is Multilingual BERT?Preprint

Pires, T., Schlinger, E., & Garrette, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.01502 · Cited by 135 · see citing works

Pre-trained Contextual Embedding of Source CodePreprint

Kanade, A., Maniatis, P., Balakrishnan, G., & Shi, K. (2019) · arXiv (Cornell University) · Cited by 40 · see citing works

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805

Deep contextualized word representations

Peters, M. E., Neumann, M., Iyyer, M., et al. (2018) · arXiv · doi:10.48550/arxiv.1802.05365

A Neural Attention Model for Abstractive Sentence Summarization

Rush, A. M., Chopra, S., & Weston, J. (2015) · arXiv · doi:10.48550/arxiv.1509.00685

oLMpics -- On what Language Model Pre-training Captures

Talmor, A., Elazar, Y., Goldberg, Y., & Berant, J. (2019) · arXiv · doi:10.48550/arxiv.1912.13283

Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks

Tai, K. S., Socher, R., & Manning, C. D. (2015) · arXiv · doi:10.48550/arxiv.1503.00075

CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation

Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C., Drain, D., Jiang, D., Tang, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.04664 · Cited by 536 · see citing works

Code Llama: Open Foundation Models for Code

Rozière, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.12950 · Cited by 531 · see citing works

Automatic Generation of Programming Exercises and Code Explanations Using Large Language Models

Sarsa, S., Denny, P., Hellas, A., & Leinonen, J. (2022) · Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 1 · doi:10.1145/3501385.3543957 · Cited by 497 · see citing works

Security and Privacy Challenges of Large Language Models: A Survey

Das, B. C., Amini, M. H., & Wu, Y. (2025) · ACM Computing Surveys · doi:10.1145/3712001 · Cited by 445 · see citing works

Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming

Kazemitabaar, M., Chow, J., Ma, C. K. T., Ericson, B. J., Weintrop, D., & Grossman, T. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580919 · Cited by 366 · see citing works

Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models

Deng, Y., Xia, C. S., Peng, H., Yang, C., & Zhang, L. (2023) · Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3597926.3598067 · Cited by 326 · see citing works

CVEfixes: automated collection of vulnerabilities and their fixes from open-source software

Bhandari, G. P., Naseer, A., & Moonen, L. (2021) · doi:10.1145/3475960.3475985 · Cited by 292 · see citing works

CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X

Zheng, Q., Xia, X., Zou, X., Dong, Y., Wang, S., Xue, Y., Shen, L., Wang, Z., Wang, A., Li, Y., et al. (2023) · Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3580305.3599790 · Cited by 263 · see citing works

ProtTrans: Towards Cracking the Language of Life's Code Through Self-Supervised Deep Learning and High Performance ComputingPreprint

Elnaggar, A., Heinzinger, M., Dallago, C., Rihawi, G., Wang, Y., Jones, L., Gibbs, T., Fehér, T., Angerer, C. M., Steinegger, M., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.06225 · Cited by 305 · see citing works

An Empirical Study of the Non-Determinism of ChatGPT in Code Generation

Ouyang, S., Zhang, J. M., Harman, M., & Wang, M. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3697010 · Cited by 258 · see citing works

“It’s Weird That it Knows What I Want”: Usability and Interactions with Copilot for Novice Programmers

Prather, J., Reeves, B. N., Denny, P., Becker, B. A., Leinonen, J., Luxton-Reilly, A., Powell, G., Finnie-Ansley, J., & Santos, E. A. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3617367 · Cited by 256 · see citing works

Few-shot training LLMs for project-specific code-summarization

Ahmed, T., & Devanbu, P. (2022) · Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3551349.3559555 · Cited by 240 · see citing works

Investigating Explainability of Generative AI for Code through Scenario-based Design

Sun, J., Liao, Q. V., Muller, M., Agarwal, M., Houde, S., Talamadupula, K., & Weisz, J. D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511119 · Cited by 219 · see citing works

CodeBLEU: a Method for Automatic Evaluation of Code Synthesis

Ren, S., Guo, D., Lu, S., Zhou, L., Liu, S., Tang, D., Sundaresan, N., Zhou, M., Blanco, A., & Ma, S. (2020) · arXiv · doi:10.48550/arxiv.2009.10297 · Cited by 214 · see citing works

InCoder: A Generative Model for Code Infilling and Synthesis

Fried, D., Aghajanyan, A., Lin, J., Wang, S., Wallace, E., Shi, F., Zhong, R., Yih, W.-T., Zettlemoyer, L., & Lewis, M. (2022) · arXiv · doi:10.48550/arxiv.2204.05999 · Cited by 187 · see citing works

Test case selection and prioritization using machine learning: a systematic literature review

Pan, R., Bagherzadeh, M., Ghaleb, T. A., & Briand, L. (2021) · Empirical Software Engineering · doi:10.1007/s10664-021-10066-6 · Cited by 197 · see citing works

Pre-trained Language Models in Biomedical Domain: A Systematic Survey

Wang, B., Xie, Q., Pei, J., Chen, Z., Tiwari, P., Li, Z., & Fu, J. (2023) · ACM Computing Surveys · doi:10.1145/3611651 · Cited by 206 · see citing works

Neural Transfer Learning for Repairing Security Vulnerabilities in C Code

Chen, Z., Kommrusch, S., & Monperrus, M. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3147265 · Cited by 174 · see citing works

Traceability Transformed: Generating More Accurate Links with Pre-Trained BERT Models

Lin, J., Liu, Y., Zeng, Q., Jiang, M., & Cleland‐Huang, J. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00040 · Cited by 167 · see citing works

In-IDE Code Generation from Natural Language: Promise and Challenges

Xu, F. F., Vasilescu, B., & Neubig, G. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3487569 · Cited by 154 · see citing works

Applying CodeBERT for Automated Program Repair of Java Simple Bugs

Mashhadi, E., & Hemmati, H. (2021) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr52588.2021.00063 · Cited by 144 · see citing works

Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code

Pan, R., Ibrahimzada, A. R., Krishna, R., Sankar, D., Wassi, L. P., Merler, M., Sobolev, B., Pavuluri, R., Sinha, S., & Jabbarvand, R. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639226 · Cited by 133 · see citing works

Contrastive Code Representation Learning

Jain, P., Jain, A., Zhang, T., Abbeel, P., Gonzalez, J., & Stoica, I. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.482 · Cited by 129 · see citing works

Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair

Wei, Y., Xia, C. S., & Zhang, L. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3616271 · Cited by 115 · see citing works

MultiPL-E: A Scalable and Polyglot Approach to Benchmarking Neural Code Generation

Cassano, F., Gouwar, J., Nguyen, D., Nguyen, S., Phipps-Costin, L., Pinckney, D., Yee, M.-H., Zi, Y., Anderson, C. J., Feldman, M. Q., et al. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3267446 · Cited by 109 · see citing works

Assessment of chemistry knowledge in large language models that generate code

White, A. D., Hocky, G. M., Gandhi, H. A., Ansari, M., Cox, S., Wellawatte, G. P., Sasmal, S., Yang, Z., Liu, K., Singh, Y., et al. (2023) · Digital Discovery · doi:10.1039/d2dd00087c · Cited by 119 · see citing works

A Survey on Software Defect Prediction Using Deep Learning

Akimova, E. N., Bersenev, A. Y., Deikov, A. A., Kobylkin, K. S., Konygin, A. V., Mezentsev, I. P., & Misilov, V. E. (2021) · Mathematics · doi:10.3390/math9111180 · Cited by 118 · see citing works

Pretrained Transformers for Text Ranking: BERT and BeyondPreprint

Lin, J., Nogueira, R. F., & Yates, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.06467 · Cited by 101 · see citing works

Deep learning-based software engineering: progress, challenges, and opportunities

Chen, X., Hu, X., Huang, Y., Jiang, H., Ji, W., Jiang, Y., Jiang, Y., Liu, B., Liu, H., Li, X., et al. (2024) · Science China Information Sciences · doi:10.1007/s11432-023-4127-5 · Cited by 114 · see citing works

What Makes Good In-Context Demonstrations for Code Intelligence Tasks with LLMs?

Gao, S., Wen, X.-C., Gao, C., Wang, W., Zhang, H., & Lyu, M. R. (2023) · 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00109 · Cited by 106 · see citing works

Automatic Semantic Augmentation of Language Model Prompts (for Code Summarization)

Ahmed, T., Pai, K. S., Devanbu, P., & Barr, E. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639183 · Cited by 112 · see citing works

An Empirical Study on Software Defect Prediction Using CodeBERT Model

Pan, C., Lu, M., & Xu, B. (2021) · Applied Sciences · doi:10.3390/app11114793 · Cited by 89 · see citing works

Recommending metamodel concepts during modeling activities with pre-trained language models

Weyssow, M., Sahraoui, H. A., & Syriani, E. (2022) · Software & Systems Modeling · doi:10.1007/s10270-022-00975-5 · Cited by 83 · see citing works

What do pre-trained code models know about code?

Karmakar, A., & Robbes, R. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678927 · Cited by 80 · see citing works

Graph-based, Self-Supervised Program Repair from Diagnostic Feedback

Yasunaga, M., & Liang, P. (2020) · arXiv · doi:10.48550/arxiv.2005.10636 · Cited by 81 · see citing works

Improving Code Summarization with Block-wise Abstract Syntax Tree Splitting

Lin, C., Ouyang, Z., Zhuang, J., Chen, J., Li, H., & Wu, R. (2021) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc52881.2021.00026 · Cited by 76 · see citing works

Code Summarization with Structure-induced Transformer

Wu, H., Zhao, H., & Zhang, M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.93 · Cited by 77 · see citing works

Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized ArchitecturesPreprint

Furrer, D., van Zee, M., Scales, N., & Schärli, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08970 · Cited by 79 · see citing works

Evaluating a large language model’s ability to solve programming exercises from an introductory bioinformatics course

Piccolo, S., Denny, P. C., Luxton-Reilly, A., Payne, S., & Ridge, P. G. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011511 · Cited by 81 · see citing works

Better Together? An Evaluation of AI-Supported Code Translation

Weisz, J. D., Muller, M., Ross, S. I., Martinez, F., Houde, S., Agarwal, M., Talamadupula, K., & Richards, J. T. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511157 · Cited by 69 · see citing works

DOBF: A Deobfuscation Pre-Training Objective for Programming LanguagesPreprint

Rozière, B., Lachaux, M.-A., Szafraniec, M., & Lample, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07492 · Cited by 96 · see citing works

A Search-Based Testing Framework for Deep Neural Networks of Source Code Embedding

Pour, M. V., Li, Z., Ma, L., & Hemmati, H. (2021) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst49551.2021.00016 · Cited by 62 · see citing works

Significant Productivity Gains through Programming with Large Language Models

Weber, T., Brandmaier, M., Schmidt, A., & Mayer, S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3661145 · Cited by 64 · see citing works

Generative Artificial Intelligence for Software Engineering—A Research Agenda

Nguyen‐Duc, A., Cabrero‐Daniel, B., Przybylek, A., Arora, C., Khanna, D., Herda, T., Rafiq, U., Melegati, J., Guerra, E., Kemell, K., et al. (2025) · Software: Practice and Experience · doi:10.1002/spe.70005 · Cited by 60 · see citing works

Systematic literature review on bias mitigation in generative AI

Afreen, J., Mohaghegh, M., & Doborjeh, M. (2025) · AI and Ethics · doi:10.1007/s43681-025-00721-9 · Cited by 66 · see citing works

Exploring Software Naturalness through Neural Language ModelsPreprint

Buratti, L., Pujar, S., Bornea, M. A., McCarley, S., Zheng, Y., Rossiello, G., Morari, A., Laredo, J., Thost, V., Zhuang, Y., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12641 · Cited by 95 · see citing works

MathBERT: A Pre-Trained Model for Mathematical Formula UnderstandingPreprint

Peng, S., Yuan, K., Gao, L., & Tang, Z. L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.00377 · Cited by 93 · see citing works

Automatic Programming: Large Language Models and Beyond

Lyu, M. R.-T., Ray, B., Roychoudhury, A., Tan, S. H., & Thongtanunam, P. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3708519 · Cited by 65 · see citing works

CLAP: Learning Transferable Binary Code Representations with Natural Language Supervision

Wang, H., Gao, Z., Zhang, C., Sha, Z., Sun, M., Zhou, Y., Zhu, W., Sun, W., Qiu, H., & Xiao, X. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3652145 · Cited by 60 · see citing works

Driving Factors of Generative AI Adoption in New Product Development Teams from a UTAUT Perspective

Xia, Y., & Chen, Y. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2375686 · Cited by 49 · see citing works

ReGVD

Nguyen, V.-A., Nguyen, D. Q., Nguyen, V., Le, T., Tran, Q. H., & Phung, D. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion55297.2022.9793807 · Cited by 183 · see citing works

Artificial intelligence generated content (AIGC) in medicine: A narrative review

Shao, L., Chen, B., Zhang, Z., Zhang, Z., & Chen, X. (2024) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2024073 · Cited by 48 · see citing works

On the Effectiveness of Transfer Learning for Code Search

Salza, P., Schwizer, C., Gu, J. N., & Gall, H. C. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3192755 · Cited by 47 · see citing works

On the transferability of pre-trained language models for low-resource programming languages

Chen, F., Fard, F. H., Lo, D., & Bryksin, T. (2022) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3524610.3527917 · Cited by 45 · see citing works

Enhanced automated code vulnerability repair using large language models

de-Fitero-Dominguez, D., Garcia-Lopez, E., Garcia-Cabot, A., & Martinez-Herraiz, J.-J. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109291 · Cited by 43 · see citing works

Making the most of small Software Engineering datasets with modern machine learning

Prenner, J. A., & Robbes, R. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3135465 · Cited by 40 · see citing works

Learning type annotation: is big data enough?

Jesse, K., Dévanbu, P., & Ahmed, T. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3473135 · Cited by 41 · see citing works

Multilingual Code Snippets Training for Program Translation

Zhu, M., Suresh, K., & Reddy, C. K. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21434 · Cited by 39 · see citing works

Leveraging automated unit tests for unsupervised code translation

Roziere, B., Zhang, J. M., Charton, F., Harman, M., Synnaeve, G., & Lample, G. (2021) · arXiv · doi:10.48550/arxiv.2110.06773 · Cited by 39 · see citing works

Bias Testing and Mitigation in LLM-based Code Generation

Huang, D., Zhang, J. M., Bu, Q., Xie, X., Chen, J., & Cui, H. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3724117 · Cited by 38 · see citing works

Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics Capacities

Ma, W., Liu, S., Zhao, M., Xie, X., Wang, W., Hu, Q., Zhang, J., & Liu, Y. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3664606 · Cited by 43 · see citing works

Untitled work

(2021) · arXiv (Cornell University) · Cited by 37 · see citing works

Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants

Sandoval, G., Pearce, H., Nys, T., Karri, R., Garg, S., & Dolan-Gavitt, B. (2022) · arXiv · doi:10.48550/arxiv.2208.09727 · Cited by 52 · see citing works

Enriching query semantics for code search with reinforcement learning

Wang, C., Nong, Z., Gao, C., Li, Z., Zeng, J., Xing, Z., & Liu, Y. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.09.025 · Cited by 36 · see citing works

ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation

Qi, W., Gong, Y., Yan, Y., Xu, C., Yao, B., Zhou, B., Cheng, B., Jiang, D., Chen, J., Zhang, R., et al. (2021) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2021.acl-demo.28 · Cited by 36 · see citing works

Semi-automated protocol disambiguation and code generation

Yen, J., Lévai, T., Ye, Q., Ren, X., Govindan, R., & Raghavan, B. (2021) · ACM SIGCOMM Conference (SIGCOMM) · doi:10.1145/3452296.3472910 · Cited by 35 · see citing works

A Toolkit for Generating Code Knowledge Graphs

Abdelaziz, I., Dolby, J., McCusker, J., & Srinivas, K. (2021) · Proceedings of the 11th Knowledge Capture Conference · doi:10.1145/3460210.3493578 · Cited by 46 · see citing works

Pre-trained Language Models in Biomedical Domain: A Systematic SurveyPreprint

Wang, B., Xie, Q., Pei, J., Chen, Z., Tiwari, P., Li, Z., & Fu, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.05006 · Cited by 34 · see citing works

Deep Neural Embedding for Software Vulnerability Discovery: Comparison and Optimization

Yuan, X., Lin, G., Tai, Y., & Zhang, J. (2022) · Security and Communication Networks · doi:10.1155/2022/5203217 · Cited by 34 · see citing works

Predicting Patch Correctness Based on the Similarity of Failing Test Cases

Tian, H., Li, Y., Pian, W., Kaboré, A. K., Liu, K., Habib, A., Klein, J., & Bissyandé, T. F. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3511096 · Cited by 33 · see citing works

Fine-Tuning Large Language Models to Improve Accuracy and Comprehensibility of Automated Code Review

Yu, Y., Rong, G., Shen, H., Zhang, H., Shao, D., Wang, M., Wei, Z., Xu, Y., & Wang, J. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695993 · Cited by 38 · see citing works

API2Com: On the Improvement of Automatically Generated Code Comments Using API Documentations

Shahbazi, R., Sharma, R., & Fard, F. H. (2021) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc52881.2021.00049 · Cited by 33 · see citing works

Code Vulnerability Detection Based on Deep Sequence and Graph Models: A Survey

Wu, B., & Zou, F. (2022) · Security and Communication Networks · doi:10.1155/2022/1176898 · Cited by 33 · see citing works

Untitled work

(2024) · doi:10.1145/3613904.3642939 · Cited by 31 · see citing works

CommitBERT: Commit Message Generation Using Pre-Trained Programming Language Model

Jung, T.-H. (2021) · NLP4PROG · doi:10.18653/v1/2021.nlp4prog-1.3 · Cited by 35 · see citing works

How could Neural Networks understand Programs?Preprint

Peng, D., Zheng, S., Li, Y., Ke, G., He, D., & Liu, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.04297 · Cited by 34 · see citing works

NER-BERT: A Pre-trained Model for Low-Resource Entity TaggingPreprint

Liu, Z., Jiang, F., Hu, Y., Chen, S., & Fung, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.00405 · Cited by 49 · see citing works

Investigating the Efficacy of Large Language Models for Code Clone Detection

Khajezade, M., Wu, J. J., Fard, F. H., Rodríguez-Pérez, G., & Shehata, M. (2024) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3643916.3645030 · Cited by 30 · see citing works

Large circuit models: opportunities and challenges

Chen, L., Chen, Y., Chu, Z., Fang, W., Ho, T.-Y., Huang, R., Huang, Y., Khan, S., Li, M., Li, X., et al. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4155-7 · Cited by 54 · see citing works

Large language models for software vulnerability detection: a guide for researchers on models, methods, techniques, datasets, and metrics

Far, S. M. T., & Feyzi, F. (2025) · International Journal of Information Security · doi:10.1007/s10207-025-00992-7 · Cited by 34 · see citing works

Learning Code-Query Interaction for Enhancing Code Searches

Li, W., Qin, H., Yan, S., Shen, B., & Chen, Y. (2020) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme46990.2020.00021 · Cited by 28 · see citing works

Stealthy Backdoors as Compression Artifacts

Tian, Y., Suya, F., Xu, F., & Evans, D. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3160359 · Cited by 27 · see citing works

Is a Single Model Enough? MuCoS: A Multi-Model Ensemble Learning Approach for Semantic Code Search

Du, L., Shi, X., Wang, Y., Shi, E., Han, S., & Zhang, D. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482127 · Cited by 28 · see citing works

A Review on Source Code Documentation

Rai, S., Belwal, R. C., & Gupta, A. K. (2022) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3519312 · Cited by 28 · see citing works

Using Large Language Models to Better Detect and Handle Software Vulnerabilities and Cyber Security ThreatsPreprint

Taghavi, S. M., & Feyzi, F. (2024) · Research Square · doi:10.21203/rs.3.rs-4387414/v1 · Cited by 32 · see citing works

Automating the removal of obsolete TODO comments

Gao, Z., Xia, X., Lo, D., Grundy, J., & Zimmermann, T. (2021) · Singapore Management University Institutional Knowledge (InK) (Singapore Management University) · Cited by 27 · see citing works

ChatGPT for Software Development: Opportunities and Challenges

Rahmaniar, W. (2024) · IT Professional · doi:10.1109/mitp.2024.3379831 · Cited by 25 · see citing works

When to Show a Suggestion? Integrating Human Feedback in AI-Assisted Programming

Mozannar, H., Bansal, G., Fourney, A., & Horvitz, E. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i9.28878 · Cited by 23 · see citing works

Self-Supervised Contrastive Learning for Code Retrieval and Summarization via Semantic-Preserving Transformations

BUI, D. Q. N., Yu, Y., & Jiang, L. (2021) · Open Research Online (The Open University) · Cited by 20 · see citing works

Machine Learning-Based Automated Grading and Feedback Tools for Programming: A Meta-Analysis

Messer, M., Brown, N. C. C., Kölling, M., & Shi, M. (2023) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3587102.3588822 · Cited by 20 · see citing works

Software Vulnerability Detection via Deep Learning over Disaggregated Code Graph RepresentationPreprint

Zhuang, Y., Suneja, S., Thost, V., Domeniconi, G., Morari, A., & Laredo, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.03341 · Cited by 24 · see citing works

ProRLearn: boosting prompt tuning-based vulnerability detection by reinforcement learning

Ren, Z., Ju, X., Chen, X., & Shen, H. (2024) · Automated Software Engineering · doi:10.1007/s10515-024-00438-9 · Cited by 20 · see citing works

Code vulnerability detection based on augmented program dependency graph and optimized CodeBERT

Zou, Z., Jiang, T., Wang, Y., Xue, T., Zhang, N., & Luan, J. (2025) · Scientific Reports · doi:10.1038/s41598-025-23029-4 · Cited by 17 · see citing works

Improving Automated Program Repair with Domain Adaptation

Zirak, A., & Hemmati, H. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3631972 · Cited by 17 · see citing works

Improving the Learning of Code Review Successive Tasks with Cross-Task Knowledge Distillation

Sghaier, O. B., & Sahraoui, H. A. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3643775 · Cited by 18 · see citing works

Deep learning for code generation: a survey

Zhang, H., Zhang, K., Li, Z., Li, J., Li, J., Li, Y., Zhao, Y., Zhu, Y. S., Liu, F., Li, G., et al. (2024) · Science China Information Sciences · doi:10.1007/s11432-023-3956-3 · Cited by 17 · see citing works

VFCFinder: Pairing Security Advisories and Patches

Dunlap, T., Lin, E., Enck, W., & Reaves, B. (2024) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3634737.3657007 · Cited by 16 · see citing works

Disentangled Code Representation Learning for Multiple Programming Languages

Zhang, J., Hong, H., Zhang⋆, Y., Wan, Y., Liu, Y., & Sui, Y. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.391 · Cited by 16 · see citing works

On using distributed representations of source code for the detection of C security vulnerabilitiesPreprint

Coimbra, D., Reis, S., Abreu, R., Păsăreanu, C. S., & Erdogmus, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.01367 · Cited by 20 · see citing works

Deep Domain Adaptation With Max-Margin Principle for Cross-Project Imbalanced Software Vulnerability Detection

Nguyen, V., Le, T., Tantithamthavorn, C. K., Grundy, J., & Phung, D. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3664602 · Cited by 15 · see citing works

Contrastive Code-Comment Pre-training

Pei, X., Liu, D., Luo, Q., & Xu, C. (2022) · Proceedings · doi:10.1109/icdm54844.2022.00050 · Cited by 2 · see citing works

NatGen: Generative pre-training by "Naturalizing" source codePreprint

Chakraborty, S., Ahmed, T., Ding, Y., Dévanbu, P., & Ray, B. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2206.07585 · Cited by 9 · see citing works

CoTexT: Multi-task Learning with Code-Text TransformerPreprint

Phan, L., Tran, H., Le, D. X., Nguyen, H., Anibal, J., Peltekian, A., & Ye, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.08645 · Cited by 7 · see citing works

CodeBERTScore: Evaluating Code Generation with Pretrained Models of Code

Zhou, S., Alon, U., Agarwal, S., & Neubig, G. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.859 · Cited by 100 · see citing works

CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation

Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.685 · Cited by 1,495 · see citing works