UniXcoder: Unified Cross-Modal Pre-training for Code Representation
Open accessAbstract
Pre-trained models for programming languages have recently demonstrated great success on code intelligence.To support both code-related understanding and generation tasks, recent works attempt to pre-train unified encoder-decoder models.However, such encoder-decoder framework is sub-optimal for auto-regressive tasks, especially code completion that requires a decoder-only manner for efficient inference.In this paper, we present UniXcoder, a unified cross-modal pre-trained model for programming language.The model utilizes mask attention matrices with prefix adapters to control the behavior of the model and leverages cross-modal contents like AST and code comment to enhance code representation.To encode AST that is represented as a tree in parallel, we propose a one-to-one mapping method to transform AST in a sequence structure that retains all structural information from the tree.Furthermore, we propose to utilize multi-modal contents to learn representation of code fragment with contrastive learning, and then align representations among programming languages using a cross-modal generation task.We evaluate UniXcoder on five code-related tasks over nine datasets.To further evaluate the performance of code fragment representation, we also construct a dataset for a new task, called zero-shot code-to-code search.Results show that our model achieves state-of-the-art performance on most tasks and analysis reveals that comment and AST can both enhance UniXcoder.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.703 · Cited by 8,821 · see citing works
SimCSE: Simple Contrastive Learning of Sentence Embeddings
Gao, T., Yao, X., & Chen, D. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.552 · Cited by 2,851 · see citing works
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., et al. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.139 · Cited by 2,824 · see citing works
Self-citationWang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.685 · Cited by 1,495 · see citing works
Self-citationWang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · arXiv · doi:10.48550/arxiv.2109.00859 · Cited by 1,495 · see citing works
Self-citationUnified Language Model Pre-training for Natural Language Understanding and Generation
Dong, L., Yang, N., Wang, W., Wei, F., Liu, X., Wang, Y., Gao, J., Zhou, M., & Hon, H.-W. (2019) · arXiv · doi:10.48550/arxiv.1905.03197 · Cited by 1,225 · see citing works
Self-citationConvolutional Neural Networks over Tree Structures for Programming Language Processing
Mou, L., Li, G., Zhang, L., Wang, T., & Jin, Z. G. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10139 · Cited by 705 · see citing works
Unified Pre-training for Program Understanding and Generation
Ahmad, W., Chakraborty, S., Ray, B., & Chang, K.-W. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.211 · Cited by 642 · see citing works
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C., Drain, D., Jiang, D., Tang, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.04664 · Cited by 536 · see citing works
Self-citationCodeSearchNet Challenge: Evaluating the State of Semantic Code Search
Husain, H., Wu, H.-H., Gazit, T., Allamanis, M., & Brockschmidt, M. (2019) · arXiv · doi:10.48550/arxiv.1909.09436 · Cited by 483 · see citing works
IntelliCode compose: code generation using transformer
Svyatkovskiy, A., Deng, S. K., Fu, S., & Sundaresan, N. (2020) · Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3368089.3417058 · Cited by 474 · see citing works
Mining source code repositories at massive scale using language modeling
Allamanis, M., & Sutton, C. (2013) · IEEE Working Conference on Mining Software Repositories · doi:10.1109/msr.2013.6624029 · Cited by 393 · see citing works
Towards a Big Data Curated Benchmark of Inter-project Code Clones
Svajlenko, J., Islam, J. F., Keivanloo, I., Roy, C. K., & Mia, M. M. (2014) · 2014 IEEE International Conference on Software Maintenance and Evolution · doi:10.1109/icsme.2014.77 · Cited by 339 · see citing works
Multi-task Learning based Pre-trained Language Model for Code Completion
Liu, F., Li, G., Zhao, Y., & Jin, Z. (2020) · Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3324884.3416591 · Cited by 188 · see citing works
Mapping Language to Code in Programmatic Context
Iyer, S., Konstas, I., Cheung, A., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1192 · Cited by 180 · see citing works
Probabilistic model for code with decision trees
Raychev, V., Bielik, P., & Vechev, M. (2016) · Conference on Object-Oriented Programming Systems, Languages, and Applications · doi:10.1145/2983990.2984041 · Cited by 157 · see citing works
SynCoBERT: Syntax-Guided Multi-Modal Contrastive Pre-Training for Code RepresentationPreprint
Wang, X. E., Wang, Y., Mi, F., Zhou, P., Wan, Y., Xiao, L., Li, L., Wu, H., Liu, J., & Jiang, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.04556 · Cited by 118 · see citing works
Self-citationExploring Software Naturalness through Neural Language ModelsPreprint
Buratti, L., Pujar, S., Bornea, M. A., McCarley, S., Zheng, Y., Rossiello, G., Morari, A., Laredo, J., Thost, V., Zhuang, Y., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.12641 · Cited by 95 · see citing works
CoSQA: 20,000+ Web Queries for Code Search and Question Answering
Huang, J., Tang, D., Shou, L., Gong, M., Xu, K., Jiang, D., Zhou, M., & Duan, N. (2021) · doi:10.18653/v1/2021.acl-long.442 · Cited by 67 · see citing works
Self-citationPre-trained Contextual Embedding of Source CodePreprint
Kanade, A., Maniatis, P., Balakrishnan, G., & Shi, K. (2019) · arXiv (Cornell University) · Cited by 40 · see citing works
(2021) · arXiv (Cornell University) · Cited by 37 · see citing works
TreeBERT: A Tree-Based Pre-Trained Model for Programming LanguagePreprint
Jiang, X., Zheng, Z., Lyu, C., Li, L., & Lyu, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.12485 · Cited by 37 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
1 of the 28 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, LIU Shujie, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, et al. 2020. Graphcode- bert: Pre-training code representations with data flow. In International Conference on Learning Rep- resentations.
- Chin-Yew Lin and Franz Josef Och. 2004. Orange: a method for evaluating automatic evaluation metrics for machine translation. In COLING 2004: Proceed- ings of the 20th International Conference on Compu- tational Linguistics, pages 501-507.
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Lan- guage models are unsupervised multitask learners. OpenAI blog, 1(8):9.
- Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909. · arXiv
Cited by
Sorted by citations
Bandi, A., Adapa, P. V. S. R., & Kuchi, Y. E. V. P. K. (2023) · Future Internet · doi:10.3390/fi15080260 · Cited by 638 · see citing works
A Survey on Large Language Models for Code Generation
Jiang, J., Wang, F., Shen, J., Kim, S., & Kim, S. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3747588 · Cited by 251 · see citing works
Automating code review activities by large-scale pre-training
Li, Z., Lu, S., Guo, D., Duan, N., Jannu, S., Jenks, G., Majumder, D., Green, J., Svyatkovskiy, A., Fu, S., et al. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549081 · Cited by 226 · see citing works
Self-citationRetrieval-Based Prompt Selection for Code-Related Few-Shot Learning
Nashid, N., Sintaha, M., & Mesbah, A. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00205 · Cited by 178 · see citing works
Self-Planning Code Generation with Large Language Models
Jiang, X., Dong, Y., Wang, L., Fang, Z., Shang, Q., Li, G., Jin, Z., & Jiao, W. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3672456 · Cited by 158 · see citing works
RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation
Zhang, F., Chen, B., Zhang, Y., Keung, J., Liu, J., Zan, D., Mao, Y., Lou, J.-G., & Chen, W. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.151 · Cited by 173 · see citing works
Exploring the Potential of ChatGPT in Automated Code Refinement: An Empirical Study
Guo, Q., Cao, J., Xie, X., Liu, S., Li, X., Chen, B., & Peng, X. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3623306 · Cited by 140 · see citing works
Ullah, S., Han, M., Pujar, S., Pearce, H. A., Coskun, A. K., & Stringhini, G. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00210 · Cited by 132 · see citing works
Vulnerability Detection with Code Language Models: How Far are We?
Ding, Y., Fu, Y., Ibrahim, O., Sitawarin, C., Chen, X., Alomair, B., Wagner, D., Ray, B., & Chen, Y. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00038 · Cited by 187 · see citing works
An Empirical Study on Fine-Tuning Large Language Models of Code for Automated Program Repair
Huang, K., Meng, X., Zhang, J., Liu, Y., Wang, W., Li, S., & Zhang, Y. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00181 · Cited by 113 · see citing works
Deep learning-based software engineering: progress, challenges, and opportunities
Chen, X., Hu, X., Huang, Y., Jiang, H., Ji, W., Jiang, Y., Jiang, Y., Liu, B., Liu, H., Li, X., et al. (2024) · Science China Information Sciences · doi:10.1007/s11432-023-4127-5 · Cited by 114 · see citing works
What Makes Good In-Context Demonstrations for Code Intelligence Tasks with LLMs?
Gao, S., Wen, X.-C., Gao, C., Wang, W., Zhang, H., & Lyu, M. R. (2023) · 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00109 · Cited by 106 · see citing works
CodeKGC: Code Language Model for Generative Knowledge Graph Construction
Bi, Z., Chen, J., Jiang, Y., Xiong, F., Guo, W., Chen, H., & Zhang, N. (2024) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3641850 · Cited by 94 · see citing works
An Empirical Comparison of Pre-Trained Models of Source Code
Niu, C., Li, C., Ng, V., Chen, D., Ge, J., & Luo, B. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00180 · Cited by 90 · see citing works
Dataflow Analysis-Inspired Deep Learning for Efficient Vulnerability Detection
Steenhoek, B. J., Gao, H., & Le, W. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3623345 · Cited by 79 · see citing works
Towards General Text Embeddings with Multi-stage Contrastive Learning
Li, Z., Zhang, X., Zhang, Y., Long, D., Xie, P., & Zhang, M. (2023) · arXiv · doi:10.48550/arxiv.2308.03281 · Cited by 79 · see citing works
Fakih, M., Dharmaji, R., Moghaddas, Y., Quirós, G., Ogundare, O., & Faruque, M. A. A. (2024) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3639477.3639743 · Cited by 85 · see citing works
Multitask-Based Evaluation of Open-Source LLM on Software Vulnerability
Yin, X., Ni, C., & Wang, S. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3470333 · Cited by 84 · see citing works
ThinkRepair: Self-Directed Automated Program Repair
Yin, X., Ni, C., Wang, S., Li, Z., Zeng, L., & Yang, X. (2024) · Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3650212.3680359 · Cited by 68 · see citing works
CodeScore: Evaluating Code Generation by Learning Code Execution
Dong, Y., Ding, J., Jiang, X., Li, G., Li, Z., & Jin, Z. G. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695991 · Cited by 76 · see citing works
Retrieving Multimodal Information for Augmented Generation: A Survey
Zhao, R., Chen, H., Wang, W., Jiao, F., Long, X., Qin, C., Ding, B., Guo, X., Li, M., Li, X., et al. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.314 · Cited by 66 · see citing works
Source Code Summarization in the Era of Large Language Models
Sun, W., Miao, Y., Li, Y., Zhang, H., Fang, C., Liu, Y., Deng, G., Liu, Y., & Chen, Z. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00034 · Cited by 95 · see citing works
Combining Fine-Tuning and LLM-Based Agents for Intuitive Smart Contract Auditing with Justifications
Ma, W., Wu, D., Sun, Y., Wang, T., Liu, S., Zhang, J., Xue, Y., & Liu, Y. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00027 · Cited by 84 · see citing works
Vulnerability Detection via Multiple-Graph-Based Code Representation
Qiu, F., Liu, Z., Hu, X., Xia, X., Chen, G., & Wang, X. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3427815 · Cited by 62 · see citing works
Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineering
Wang, R., Guo, J., Gao, C., Fan, G., Chong, C. Y., & Xia, X. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3728963 · Cited by 62 · see citing works
CoCoSoDa: Effective Contrastive Learning for Code Search
Shi, E.-S., Wang, Y.-L., Gu, W., Du, L., Zhang, H.-Y., Han, S., Zhang, D.-M., & Sun, H. (2022) · International Conference on Software Engineering · doi:10.1109/icse48619.2023.00185 · Cited by 89 · see citing works
Self-citationPre-Trained Model-Based Automated Software Vulnerability Repair: How Far are We?
Zhang, Q., Fang, C., Yu, B., Sun, W., Zhang, T., & Chen, Z. (2023) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2023.3308897 · Cited by 60 · see citing works
Language Models for Code Completion: A Practical Evaluation
Izadi, M., Katzy, J., van Dam, T., Otten, M., Popescu, R. M., & Arie Van Deursen, A. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639138 · Cited by 69 · see citing works
CCTEST: Testing and Repairing Code Completion Systems
Li, Z., Wang, C., Liu, Z., Wang, H., Chen, D., Wang, S., & Gao, C. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00110 · Cited by 62 · see citing works
Prompt Engineering or Fine-Tuning: An Empirical Assessment of LLMs for Code
Shin, J., Tang, C. J., Mohati, T., Nayebi, M., Wang, S., & Hemmati, H. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00082 · Cited by 101 · see citing works
Semantic feature learning for software defect prediction from source code and external knowledge
Liu, J., Ai, J., Lu, M., Wang, J., & Shi, H. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111753 · Cited by 54 · see citing works
Gamma: Revisiting Template-Based Automated Program Repair Via Mask Prediction
Zhang, Q., Fang, C., Zhang, T., Yu, B., Sun, W., & Chen, Z. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00063 · Cited by 57 · see citing works
A review on multimodal zero‐shot learning
Cao, W., Wu, Y., Sun, Y., Zhang, H., Ren, J., Gu, D., & Wang, X. (2023) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1488 · Cited by 50 · see citing works
One Adapter for All Programming Languages? Adapter Tuning for Code Search and Summarization
Wang, D., Chen, B., Li, S., Luo, W., Peng, S., Dong, W., & Liao, X. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00013 · Cited by 47 · see citing works
Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attacks
Cotroneo, D., Improta, C., Liguori, P., & Natella, R. (2024) · Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension · doi:10.1145/3643916.3644416 · Cited by 45 · see citing works
Ni, C., Yin, X., Yang, K., Zhao, D., Xing, Z., & Xia, X. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616358 · Cited by 44 · see citing works
Enhanced automated code vulnerability repair using large language models
de-Fitero-Dominguez, D., Garcia-Lopez, E., Garcia-Cabot, A., & Martinez-Herraiz, J.-J. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109291 · Cited by 43 · see citing works
SCL-CVD: Supervised contrastive learning for code vulnerability detection via GraphCodeBERT
Wang, R., Xu, S., Tian, Y., Ji, X., Sun, X., & Jiang, S. (2024) · Computers & Security · doi:10.1016/j.cose.2024.103994 · Cited by 43 · see citing works
Shi, J., Yang, Z., & Lo, D. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3708525 · Cited by 43 · see citing works
Automatic Commit Message Generation: A Critical Review and Directions for Future Work
Zhang, Y., Qiu, Z., Stol, K.-J., Zhu, W., Zhu, J., Tian, Y., & Liu, H. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3364675 · Cited by 42 · see citing works
Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics Capacities
Ma, W., Liu, S., Zhao, M., Xie, X., Wang, W., Hu, Q., Zhang, J., & Liu, Y. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3664606 · Cited by 43 · see citing works
An Empirical Study of Retrieval-Augmented Code Generation: Challenges and Opportunities
Yang, Z., Chen, S., Gao, C., Li, Z., Hu, X., Liu, K., & Xia, X. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3717061 · Cited by 37 · see citing works
Kayumova, K., Akbarova, S. K., Bobokeldiyeva, M., Abdukarimova, G., Pulatovna, K. U., & Xasanova, Z. (2024) · International Conference on Future Networks and Distributed Systems (ICFNDS) · doi:10.1145/3726122.3726228 · Cited by 37 · see citing works
Generative Type Inference for Python
Peng, Y. F., Wang, C., Wang, W., Gao, C., & Lyu, M. R.-T. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00031 · Cited by 36 · see citing works
Towards Causal Deep Learning for Vulnerability Detection
Rahman, M. M., Ceka, I., Mao, C., Chakraborty, S., Ray, B., & Le, W. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639170 · Cited by 36 · see citing works
Domain Adaptive Code Completion via Language Models and Decoupled Domain Databases
Tang, Z. J., Ge, J., Liu, S., Zhu, T., Xu, T., Huang, L., & Luo, B. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00076 · Cited by 40 · see citing works
Agents in software engineering: survey, landscape, and vision
Wang, Y., Zhong, W., Huang, Y., Shi, E., Yang, M., Chen, J., Li, H., Ma, Y., Wang, Q., & Zheng, Z. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00544-2 · Cited by 50 · see citing works
Self-citationTang, M., Tang, W., Gui, Q., Hu, J., & Zhao, M. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122216 · Cited by 33 · see citing works
Exploring the Effectiveness of LLMs in Automated Logging Statement Generation: An Empirical Study
Li, Y., Huo, Y., Jiang, Z., Zhong, R., He, P., Su, Y., Briand, L., & Lyu, M. R.-T. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3475375 · Cited by 32 · see citing works
Pre-training by Predicting Program Dependencies for Vulnerability Analysis Tasks
Liu, Z., Tang, Z., Zhang, J., Xia, X., & Yang, X. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639142 · Cited by 31 · see citing works
Multi-target Backdoor Attacks for Code Pre-trained Models
Li, Y., Liu, S., Chen, K., Xie, X., Zhang, T., & Liu, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.399 · Cited by 37 · see citing works
APPT: Boosting Automated Patch Correctness Prediction via Fine-Tuning Pre-Trained Models
Zhang, Q., Fang, C., Sun, W., Liu, Y., He, T., Hao, X., & Chen, Z. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3354969 · Cited by 35 · see citing works
CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search
Li, X., Gong, Y., Shen, Y., Qiu, X., Zhang, H., Yao, B., Qi, W., Jiang, D., Chen, W., & Duan, N. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.187 · Cited by 31 · see citing works
Self-citationWho evaluates the evaluators? On automatic metrics for assessing AI-based offensive code generators
Liguori, P., Improta, C., Natella, R., Čukić, B., & Cotroneo, D. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120073 · Cited by 31 · see citing works
Comprehensive Fine-Tuning Large Language Models of Code for Automated Program Repair
Huang, K., Zhang, J., Bao, X., Wang, X., & Liu, Y. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3532759 · Cited by 30 · see citing works
RLCoder: Reinforcement Learning for Repository-Level Code Completion
Wang, Y., Wang, Y., Guo, D., Chen, J., Zhang, R., Ma, Y., & Zheng, Z. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00014 · Cited by 43 · see citing works
Self-citationCode Execution with Pre-trained Language Models
Liu, C., Lu, S., Chen, W., Jiang, D., Svyatkovskiy, A., Fu, S., Sundaresan, N., & Duan, N. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.308 · Cited by 27 · see citing works
Self-citationCodepori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology
Rasheed, Z., Malik, S., Kemell, K.-K., Muhammad, W., Saari, M., Systä, K., & Abrahamsson, P. (2024) · doi:10.2139/ssrn.4979510 · Cited by 27 · see citing works
How Important Are Good Method Names in Neural Code Generation? A Model Robustness Perspective
Yang, G., Zhou, Y., Yang, W., Yue, T., Chen, X., & Chen, T. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3630010 · Cited by 28 · see citing works
ChatAssert: LLM-Based Test Oracle Generation With External Tools Assistance
Hayet, I., Scott, A., & d’Amorim, M. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3519159 · Cited by 27 · see citing works
An Empirical Study of Parameter-Efficient Fine-Tuning Methods for Pre-Trained Code Models
Liu, J., Sha, C., & Peng, X. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00125 · Cited by 26 · see citing works
Delving into Parameter-Efficient Fine-Tuning in Code Change Learning: An Empirical Study
Liu, S., Keung, J., Yang, Z., Liu, F., Zhou, Q., & Liao, Y. (2024) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner60148.2024.00055 · Cited by 36 · see citing works
Survey of Code Search Based on Deep Learning
Xie, Y., Lin, J., Dong, H., Zhang, L., & Wu, Z. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3628161 · Cited by 27 · see citing works
Exploring Automated Assertion Generation via Large Language Models
Zhang, Q., Sun, W., Fang, C., Yu, B., Li, H., Yan, M. L., Zhou, J., & Chen, Z. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3699598 · Cited by 24 · see citing works
Computational Linguistics and Big Data in Knowledge Management
Shokirova, D., Ulmasbaeva, M. A., Abdusaidova, G., Rozmatovna, U. O., Kozimovna, T. Z., & Umarova, R. (2024) · International Conference on Future Networks and Distributed Systems (ICFNDS) · doi:10.1145/3726122.3726240 · Cited by 24 · see citing works
RefBERT: A Two-Stage Pre-trained Framework for Automatic Rename Refactoring
Liu, H., Wang, Y., Zhao, W., Xu, Y., Wang, J., Li, H., & Ji, R. (2023) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3597926.3598092 · Cited by 25 · see citing works
Self-citationEsale: Enhancing Code-Summary Alignment Learning for Source Code Summarization
Fang, C.-R., Sun, W.-S., Chen, Y., Chen, X., Wei, Z., Zhang, Q.-J., You, Y., Luo, B., Liu, Y., & Chen, Z.-Y. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3422274 · Cited by 34 · see citing works
SCALE: Constructing Structured Natural Language Comment Trees for Software Vulnerability Detection
Wen, X.-C., Gao, C., Gao, S., Xiao, Y., & Lyu, M. R.-T. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3652124 · Cited by 23 · see citing works
Hands-on analysis of using large language models for the auto evaluation of programming assignments
Mohamed, K. M., Yousef, M., Medhat, W., Mohamed, E. H., Khoriba, G. A., & Arafa, T. (2024) · Information Systems · doi:10.1016/j.is.2024.102473 · Cited by 26 · see citing works
Distinguishing LLM-Generated from Human-Written Code by Contrastive Learning
Xu, X., Ni, C., Guo, X., Liu, S., Wang, X., Liu, K., & Yang, X. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3705300 · Cited by 26 · see citing works
Espejel, J. L., Alassan, M. S. Y., Chouham, E. M., Dahhane, W., & Ettifouri, E. H. (2023) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100013 · Cited by 23 · see citing works
Non-Autoregressive Line-Level Code Completion
Liu, F., Fu, Z., Li, G., Jin, Z., Liu, H., Hao, Y., & Zhang, L. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3649594 · Cited by 22 · see citing works
FlakyFix: Using Large Language Models for Predicting Flaky Test Fix Categories and Test Code Repair
Fatima, S., Hemmati, H., & Briand, L. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3472476 · Cited by 22 · see citing works
Hybrid Automated Program Repair by Combining Large Language Models and Program Analysis
Li, F., Jiang, J., Sun, J., & Zhang, H. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3715004 · Cited by 30 · see citing works
BTLink : automatic link recovery between issues and commits based on pre-trained BERT model
Lan, J., Gong, L., Zhang, J., & Zhang, H. (2023) · Empirical Software Engineering · doi:10.1007/s10664-023-10342-7 · Cited by 22 · see citing works
A Needle is an Outlier in a Haystack: Hunting Malicious PyPI Packages with Code Clustering
Liang, W., Ling, X. G., Wu, J., Luo, T., & Wu, Y. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00085 · Cited by 22 · see citing works
Function-Level Vulnerability Detection Through Fusing Multi-Modal Knowledge
Ni, C., Guo, X., Zhu, Y., Xu, X., & Yang, X. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00084 · Cited by 22 · see citing works
Hussain, S., Nadeem, M., Baber, J., Hamdi, M., Rajab, A. D., Reshan, M. S. A., & Shaikh, A. (2024) · Scientific Reports · doi:10.1038/s41598-024-56871-z · Cited by 21 · see citing works
AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodology
Nguyen, M. H., Chau, T. P., Nguyen, P. X., & Bui, N. D. Q. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00026 · Cited by 34 · see citing works
Wang, J., Xiao, H., Zhong, S., & Xiao, Y. (2023) · Future Generation Computer Systems · doi:10.1016/j.future.2023.05.016 · Cited by 21 · see citing works
Backdooring Neural Code Search
Sun, W., Chen, Y., Tao, G., Fang, C., Zhang, X., Zhang, Q., & Luo, B. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.540 · Cited by 24 · see citing works
Automatically Recommend Code Updates: Are We There Yet?
Liu, Y., Tantithamthavorn, C. K., Liu, Y., Thongtanunam, P., & Li, L. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3678167 · Cited by 22 · see citing works
Automated Commit Message Generation With Large Language Models: An Empirical Study and Beyond
Xue, P., Wu, L., Yu, Z., Jin, Z. Y., Yang, Z., Li, X., Yang, Z., & Tan, Y. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3478317 · Cited by 35 · see citing works
The “Code” of Ethics: A Holistic Audit of AI Code Generators
Ma, W., Song, Y., Xue, M., Wen, S., & Xiang, Y. (2024) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2024.3367737 · Cited by 21 · see citing works
Transformers in source code generation: A comprehensive survey
Ghaemi, H., Alizadehsani, Z., Shahraki, A., & Corchado, J. M. (2024) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2024.103193 · Cited by 23 · see citing works
When to Stop? Towards Efficient Code Generation in LLMs with Excess Token Prevention
Guo, L., Wang, Y., Shi, E., Zhong, W., Zhang, H., Chen, J., Zhang, R., Ma, Y., & Zheng, Z. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3680343 · Cited by 20 · see citing works
Self-citationDong, Y., Tang, Y., Cheng, X., & Yang, Y. (2023) · Information and Software Technology · doi:10.1016/j.infsof.2023.107290 · Cited by 19 · see citing works
Quantivine: A Visualization Approach for Large-scale Quantum Circuit Representation and Analysis
Wen, Z., Liu, Y., Tan, S., Chen, J., Zhu, M., Han, D., Yin, J., Xu, M., & Chen, W. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3327148 · Cited by 21 · see citing works
Self-citationOn the Evaluation of Neural Code Translation: Taxonomy and Benchmark
Jiao, M., Yu, T., Li, X., Qiu, G., Gu, X., & Shen, B. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00114 · Cited by 20 · see citing works
Pre-training Code Representation with Semantic Flow Graph for Effective Bug Localization
Du, Y., & Yu, Z. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3616338 · Cited by 19 · see citing works
Ridoy, S. Z., Shaon, M. S. H., Cuzzocrea, A., & Akter, M. S. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825609 · Cited by 18 · see citing works
Survey of Natural Language Processing for Education: Taxonomy, Systematic Review, and Future Trends
Lan, Y., Li, X., Du, H., Lu, X., Gao, M. Z., Qian, W., & Zhou, A. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3621181 · Cited by 43 · see citing works
Exploring Representation-level Augmentation for Code Search
HaoChen, L., Miao, C., Leung, C., Huang, Y., Huang, Y., Zhang, H., & Wang, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.327 · Cited by 18 · see citing works
Self-citationDing, Z.-S., Tang, Y., Cheng, X., Li, H., & Shang, W.-Y. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3624740 · Cited by 29 · see citing works
EALink: An Efficient and Accurate Pre-Trained Framework for Issue-Commit Link Recovery
Zhang, C., Wang, Y., Zhao, W., Xu, Y., Wang, J., Li, H., & Ji, R. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00059 · Cited by 18 · see citing works
Self-citationPTM-APIRec: Leveraging Pre-trained Models of Source Code in API Recommendation
Li, Z., Li, C., Tang, Z. J., Huang, W., Ge, J., Luo, B., Ng, V., Wang, T., Hu, Y., & Zhang, X. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3632745 · Cited by 18 · see citing works
Gao, S., Mao, W., Gao, C., Li, L., Hu, X., Xia, X., & Lyu, M. R.-T. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639216 · Cited by 18 · see citing works
Code Summarization Beyond Function Level
Makharev, V., & Ivanov, V. V. (2025) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1109/llm4code66737.2025.00024 · Cited by 19 · see citing works
Zhu, T., Li, Z., Pan, M., Shi, C., Zhang, T., Pei, Y., & Li, X. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3631975 · Cited by 17 · see citing works
Parameter-Efficient Multi-classification Software Defect Detection Method Based on Pre-trained LLMs
Wang, X., Lu, L., Yang, Z., Tian, Q., & Lin, H. (2024) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-024-00551-3 · Cited by 17 · see citing works
- Authors
- Guo, D., Lu, S., Duan, N., Wang, Y., Zhou, M., & Yin, J.
- Affiliations
- Sun Yat-sen University (CN)
- Microsoft Research Asia (China) (CN)
- Venue
- Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2022
- Pages
- 7212-7225
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W4221166942
- Cited by
- 673
- References
- 28
- Citations by year
- 2026: 153 · 2025: 239 · 2024: 148 · 2023: 122 · 2022: 11
- FWCI
- 116.24
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Funders
- National Natural Science Foundation of China (U1811262, U1811261, U2001211, U1811264, U1911203)
- Basic and Applied Basic Research Foundation of Guangdong Province (2019B15 15130001)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Keywords
- UniXcoder, code representation, programming language models, code comments, contrastive learning, cross-modal generation, code completion, code intelligence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
