The DOI matches its DataCite record; 39 of the 39 references it cites were found in a registry or an index.
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Open accessAbstract
We present CodeBERT, a bimodal pre-trained model for programming language (PL) and nat-ural language (NL). CodeBERT learns general-purpose representations that support downstream NL-PL applications such as natural language codesearch, code documentation generation, etc. We develop CodeBERT with Transformer-based neural architecture, and train it with a hybrid objective function that incorporates the pre-training task of replaced token detection, which is to detect plausible alternatives sampled from generators. This enables us to utilize both bimodal data of NL-PL pairs and unimodal data, where the former provides input tokens for model training while the latter helps to learn better generators. We evaluate CodeBERT on two NL-PL applications by fine-tuning model parameters. Results show that CodeBERT achieves state-of-the-art performance on both natural language code search and code documentation generation tasks. Furthermore, to investigate what type of knowledge is learned in CodeBERT, we construct a dataset for NL-PL probing, and evaluate in a zero-shot setting where parameters of pre-trained models are fixed. Results show that CodeBERT performs better than previous pre-trained models on NL-PL probing.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Flagged first, then by citations
Research works printed but not linked: 2
- Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. { ELECTRA } : Pretraining text encoders as discriminators rather than generators. In International Conference on Learning Representations .
- Dan Jurafsky. 2000. Speech & language processing . Pearson Education India.
Cited by
Sorted by citations
Items 101–200 of 276
Disentangled Code Representation Learning for Multiple Programming Languages
Zhang, J., Hong, H., Zhang⋆, Y., Wan, Y., Liu, Y., & Sui, Y. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.391 · Cited by 16
On using distributed representations of source code for the detection of C security vulnerabilitiesPreprint
Coimbra, D., Reis, S., Abreu, R., Păsăreanu, C. S., & Erdogmus, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.01367 · Cited by 20
Nguyen, V., Le, T., Tantithamthavorn, C. K., Grundy, J., & Phung, D. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3664602 · Cited by 22
Source Code Vulnerability Detection Based on Joint Graph and Multimodal Feature Fusion
Jin, D., He, C., Zou, Q., Qin, Y., & Wang, B. (2025) · Electronics · doi:10.3390/electronics14050975 · Cited by 15
Code Recommendation for Open Source Software Developers
Jin, Y., Bai, Y., Zhu, Y., Sun, Y., & Wang, W. W. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583503 · Cited by 15
MulCode: A Multi-task Learning Approach for Source Code Understanding
Wang, D., Yu, Y., Li, S., Dong, W., Wang, J., & Liao, Q. (2021) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner50967.2021.00014 · Cited by 15
Neural Code Search Revisited: Enhancing Code Snippet Retrieval through Natural Language IntentPreprint
Heyman, G., & Cutsem, T. V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.12193 · Cited by 29
CORAL: COde RepresentAtion learning with weakly-supervised transformers for analyzing data analysis
Zhang, G., Merrill, M. A., Liu, Y., Heer, J., & Althoff, T. (2022) · EPJ Data Science · doi:10.1140/epjds/s13688-022-00327-9 · Cited by 15
Co-training for Commit Classification
Lee, J. Y. D., & Chieu, H. L. (2021) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/2021.wnut-1.43 · Cited by 15
Semantic-Enriched Code Knowledge Graph to Reveal Unknowns in Smart Contract Code Reuse
Huang, Q. Y., Liao, D., Xing, Z., Zuo, Z., Wang, C., & Xia, X. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3597206 · Cited by 14
Context-based statement-level vulnerability localization
Nguyen, T.-T., & Vo, H. D. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107406 · Cited by 14
FastSpec: Scalable Generation and Detection of Spectre Gadgets Using Neural Embeddings
Tol, M. C., Gülmezoğlu, B., Yurtseven, K., & Sunar, B. (2021) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp51992.2021.00047 · Cited by 16
Jin, X., & Lin, Z. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3660769 · Cited by 17
Learning to Find Usages of Library Functions in Optimized Binaries
Ahmed, T., Dévanbu, P., & Sawant, A. A. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3106572 · Cited by 16
Vulnerabilities and Security Patches Detection in OSS: A Survey
Lin, R., Fu, Y., Yi, W., Yang, J., Cao, J. P., Dong, Z., Xie, F., & Li, H. (2024) · ACM Computing Surveys · doi:10.1145/3694782 · Cited by 22
CodeQA: A Question Answering Dataset for Source Code Comprehension
Liu, C., & Wan, X. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.223 · Cited by 18
RepoTransBench: A Real-World Multilingual Benchmark for Repository-Level Code Translation
Wang, Y., Wang, Y., Wang, S., Guo, D., Chen, J., Grundy, J., Liu, X., Ma, Y., Mao, M., Zhang, H., et al. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3645056 · Cited by 12
Self-citationWhat Makes a Good TODO Comment?
Wang, H., Gao, Z., Bi, T., Grundy, J., Wang, X., Wu, M., & Yang, X. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3664811 · Cited by 12
Long-Range Modeling of Source Code Files with eWASH: Extended Window Access by Syntax Hierarchy
Clement, C., Lu, S., Liu, X., Tufano, M., Drain, D., Duan, N., Sundaresan, N., & Svyatkovskiy, A. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.387 · Cited by 12
Self-citationAn ALBERT-based Similarity Measure for Mathematical Answer Retrieval
Reusch, A., Thiele, M., & Lehner, W. (2021) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3404835.3463023 · Cited by 13
Fine-Tuning GPT-2 to Patch Programs, Is It Worth It?
Lajkó, M., Horváth, D., Csuvik, V., & Vidács, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-10542-5_6 · Cited by 11
Searching for Quality: Genetic Algorithms and Metamorphic Testing for Software Engineering ML
Applis, L., Panichella, A., & Marang, R. (2023) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/3583131.3590379 · Cited by 16
Translating Natural Language Instructions to Computer Programs for Robot Manipulation
Venkatesh, S. G., Upadrashta, R., & Amrutur, B. S. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636342 · Cited by 13
PlotCoder: Hierarchical Decoding for Synthesizing Visualization Code in Programmatic Context
Chen, X., Gong, L., Cheung, A., & Song, D. X. (2021) · doi:10.18653/v1/2021.acl-long.169 · Cited by 13
RAG or Fine-tuning? A Comparative Study on LCMs-based Code Completion in Industry
Wang, C., Yang, Z., Gao, S., Gao, C., Peng, T., Huang, H., Deng, Y., & Lyu, M. R.-T. (2025) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3696630.3728535 · Cited by 13
Position Papers for the ASCR Workshop on Basic Research Needs in Quantum Computing and Networking
Lougovski, P., Parekh, O., Broz, J., Byrd, M., Chembo, Y. K., de Jong, W. A., Figueroa, E. V., Humble, T. S., Larson, J. M., Quiroz, G., et al. (2023) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · doi:10.2172/2283320 · Cited by 10
ReACC: A Retrieval-Augmented Code Completion Framework
Lu, S., Duan, N., Han, H., Guo, D., Hwang, S.-W., & Svyatkovskiy, A. (2022) · arXiv · doi:10.48550/arxiv.2203.07722 · Cited by 96
Self-citationTian, J.-H., Zhang, L., & Lian, X. (2023) · Mathematics · doi:10.3390/math11030623 · Cited by 11
Alakmeh, T., Reich, D. R., Jäger, L. A., & Fritz, T. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3660795 · Cited by 13
Vibe Coding in Practice: Building a Driving Simulator Without Expert Programming Skills
Fortes-Ferreira, M., Alam, M. S., & Bazilinskyy, P. (2025) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/3744335.3758482 · Cited by 9
Shedding Light on Software Engineering-specific Metaphors and Idioms
Imran, M. M., Chatterjee, P., & Damevski, K. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639585 · Cited by 11
Knowledge Tracing in Programming Education Integrating Students’ Questions
Kim, D., Kim, S., & Jo, Y. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.1343 · Cited by 9
Bisztray, T., Cherif, B., Dubniczky, R. A., Gruschka, N., Borsos, B., Ferrag, M. A., Kovács, A., Mavroeidis, V., & Tihanyi, N. (2025) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3733799.3762964 · Cited by 11
Software Defect Prediction via Generative Adversarial Networks and Pre-Trained Model
Song, W., Gan, L., & Bao, T. (2024) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2024.01503119 · Cited by 8
Contextualized Code Representation Learning for Commit Message GenerationPreprint
Nie, L. Y., Gao, C., Zhong, Z., Lam, W. P., Liu, Y., & Xu, Z. (2020) · arXiv (Cornell University) · Cited by 8
Semantic code search using Code2Vec: A bag-of-paths model
Arumugam, L. (2020) · UWSpace (University of Waterloo) · Cited by 8
Literature review on vulnerability detection using NLP technologyPreprint
Wu, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.11230 · Cited by 13
SYNFIX: Automatically Fixing Syntax Errors using Compiler DiagnosticsPreprint
Ahmed, T., Ledesma, N. R., & Dévanbu, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.14671 · Cited by 18
Cascaded Fast and Slow Models for Efficient Semantic Code SearchPreprint
Gotmare, A. D., Li, J., Joty, S., & Hoi, S. C. H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.07811 · Cited by 10
Bert-Based Feature Extraction for Long-Lived Bug Prediction in Floss: A Comparative StudyPreprint
Gomes, L. A. F., Côrtes, M. L., & Torres, R. D. S. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4166555 · Cited by 8
Machine Learning for Actionable Warning Identification: A Comprehensive Survey
Ge, X., Fang, C., Li, X., Sun, W., Wu, D., Zhai, J., Lin, S., Zhao, Z., Liu, Y., & Chen, Z. (2024) · ACM Computing Surveys · doi:10.1145/3696352 · Cited by 7
ProtoTransformer: A Meta-Learning Approach to Providing Student FeedbackPreprint
Wu, M., Goodman, N. D., Piech, C., & Finn, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.14035 · Cited by 24
Leveraging Large Language Models to Assist Teachers in Code Grading
Cipriano, E., Ferrato, A., Limongelli, C., Schicchi, D., & Taibi, D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98459-4_15 · Cited by 6
Chen, T., Li, L., Shan, B., Liang, G., Li, D., Wang, Q., & Xie, T. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3717060 · Cited by 13
Interpretable Deep Learning for Efficient Code Smell Prioritization in Software Development
Rashid, M. M., Osman, M. H., Sharif, K. Y., & Zulzalil, H. B. (2025) · IEEE Access · doi:10.1109/access.2025.3543277 · Cited by 8
Jointly Learning to Repair Code and Generate Commit Message
Bai, J., Zhou, L., Blanco, A., Liu, S., Wei, F., Zhou, M., & Li, Z. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.771 · Cited by 6
Self-citationFLAME: A Small Language Model for Spreadsheet Formulas
Joshi, H., Ebenezer, A., Cambronero, J., Gulwani, S., Kanade, A., Le, V., Radiček, I., & Verbruggen, G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i12.29197 · Cited by 12
A Simple Approach for Handling Out-of-Vocabulary Identifiers in Deep Learning for Source Code
Chirkova, N., & Troshin, S. (2020) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.naacl-main.26 · Cited by 12
Qi, W., Gong, Y., Yan, Y., Xu, C., Yao, B., Zhou, B., Cheng, B., Jiang, D., Chen, J., Zhang, R., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08006 · Cited by 27
Self-citationImproving VulRepair’s Perfect Prediction by Leveraging the LION Optimizer
Kishiyama, B., Lee, Y., & Yang, J. Y. (2024) · Applied Sciences · doi:10.3390/app14135750 · Cited by 5
Is Attention All You Need? Toward a Conceptual Model for Social Awareness in Large Language Models
Voria, G., Catolino, G., & Palomba, F. (2024) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1145/3650105.3652294 · Cited by 7
Measurement of Embedding Choices on Cryptographic API Completion Tasks
Xiao, Y., Song, W., Ahmed, S., Ge, X., Viswanath, B., Meng, N., & Yao, D. D. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3625291 · Cited by 5
PyTorrent: A Python Library Corpus for Large-scale Language ModelsPreprint
Bahrami, M., Shrikanth, N. C., Ruangwan, S., Liu, L., Mizobuchi, Y., Fukuyori, M., Chen, W., Munakata, K., & Menzies, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.01710 · Cited by 18
FACOS: Finding API Relevant Contents on Stack Overflow with Semantic and Syntactic AnalysisPreprint
Luong, K., Hadi, M., Thung, F., Fard, F. H., & Lo, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.07238 · Cited by 6
Six-Tier Architecture for AI-Generated Software Development: A Large Language Models ApproachPreprint
Uzair, W., & Naz, S. (2023) · Research Square · doi:10.21203/rs.3.rs-3086026/v1 · Cited by 5
Occlusion-Based Detection of Trojan-Triggering Inputs in Large Language Models of CodePreprint
Hussain, A., Rabin, M. R. I., Ahmed, T., Alipour, M. A., & Xu, B. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4798300 · Cited by 14
LongCodeZip: Compress Long Context for Code Language Models
Shi, Y., Qian, Y., Zhang, H., Shen, B., & Gu, X. (2025) · 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00020 · Cited by 7
DMVL4AVD: a deep multi-view learning model for automated vulnerability detection
Du, X., Zhou, Y., & Du, H. (2025) · Neural Computing and Applications · doi:10.1007/s00521-024-10892-x · Cited by 5
Source Code Plagiarism Detection with Pre-Trained Model Embeddings and Automated Machine Learning
Ebrahim, F., & Joy, M. S. (2023) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-092-2_034 · Cited by 5
Thematic Analysis of Expert Opinions on the Use of Large Language Models in Software Development
Yadav, S., Kaushik, A., & Qureshi, A. M. (2025) · Applied AI Letters · doi:10.1002/ail2.127 · Cited by 5
AttnCall: Refining Indirect Call Targets in Binaries with Attention
Sun, R., Guo, Y., Wang, Z., & Zeng, Q. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-51482-1_20 · Cited by 4
Knowledge-augmented Methods for Natural Language Generation
Jiang, M., Lin, B. Y., Wang, S., Yi‐chong, X., Yu, W., & Zhu, C. (2024) · SpringerBriefs in computer science · doi:10.1007/978-981-97-0747-8_4 · Cited by 4
Evaluating the Effectiveness of Deep Learning Models for Foundational Program Analysis Tasks
Chen, Q., Yu, C., Liu, R., Zhang, C., Wang, Y., Wang, K., Su, T., & Wang, L. (2024) · Proceedings of the ACM on Programming Languages · doi:10.1145/3649829 · Cited by 4
MCRepair: Multi-Chunk Program Repair via Patch Optimization with Buggy Block
Kim, J., & Lee, B. (2023) · ACM Symposium on Applied Computing · doi:10.1145/3555776.3577762 · Cited by 4
SIFAST: An Efficient Unix Shell Embedding Framework for Malicious Detection
Chen, S., Yang, R., Zhang, H., Wu, H., Zheng, Y., Fu, X., & Liu, Q. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-49187-0_4 · Cited by 4
Yet another combination of IR- and neural-based comment generation
Huang, Y., Wei, M., Wang, S., Wang, J., & Wang, Q. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2022.107001 · Cited by 9
AlgoLabel: A Large Dataset for Multi-Label Classification of Algorithmic Challenges
Iacob, R. C. A., Monea, V. C., Radulescu, D. C., Ceapă, A.-F., Rebedea, T. E., & Trăușan-Matu, Ș. (2020) · Mathematics · doi:10.3390/math8111995 · Cited by 5
Is a Single Model Enough? MuCoS: A Multi-Model Ensemble Learning for Semantic Code SearchPreprint
Du, L., Shi, X., Wang, Y., Shi, E., Han, S., & Zhang, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.04773 · Cited by 4
GPT-4 vs. GPT-3.5 AS CODING ASSISTANTSPreprint
Moussiades, L., Zografos, G., & Papakostas, G. A. (2024) · Research Square · doi:10.21203/rs.3.rs-3920214/v1 · Cited by 4
On the Impacts of Contexts on Repository-Level Code Generation
Hai, N. L., Nguyen, D. M., & Bui, N. D. Q. (2024) · arXiv · doi:10.48550/arxiv.2406.11927 · Cited by 3
Zhou, X., Weyssow, M., Widyasari, R., Zhang, T., He, J., Lyu, Y., Chang, J., Zhang, B., Huang, D., & Lo, D. (2025) · arXiv · doi:10.48550/arxiv.2502.06215 · Cited by 3
The triple attention transformer: advancing contextual coherence in transformer models
Ghaith, S. (2024) · Evolutionary Intelligence · doi:10.1007/s12065-024-00954-3 · Cited by 4
Yu, Y., & Zhang, C. (2025) · International Conference on Big Data, Information and Computer Network (BDICN) · doi:10.1145/3727353.3727471 · Cited by 3
Learning to represent code semantics
Liu, F., Li, G., Zhao, Q., & Zhang, L. (2025) · Science China Information Sciences · doi:10.1007/s11432-023-3898-5 · Cited by 3
Metamorphic Testing of Deep Code Models: A Systematic Literature Review
Asgari, A., de Koning, M., Derakhshanfar, P., & Panichella, A. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3766552 · Cited by 7
A Comparison of Different Source Code Representation Methods for Vulnerability Prediction in Python
Bagheri, A., & Hegedűs, P. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-85347-1_20 · Cited by 31
Can Neural Decompilation Assist Vulnerability Prediction on Binary Code?
Cotroneo, D., Grasso, F. C., Natella, R., & Orbinato, V. (2025) · European Workshop on System Security · doi:10.1145/3722041.3723097 · Cited by 3
Question Generation from Code Snippets and Programming Error Messages
Yao, B., Chen, W., Gong, Y., Zhou, B., Xie, J., Wei, Z., Cheng, B., & Duan, N. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-88480-2_32 · Cited by 3
Self-citationAutomating software size measurement from python code using language models
Tenekeci, S., Ünlü, H., Gül, B. A., Keleş, D., Küçük, M., & Demirörs, O. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00571-z · Cited by 3
Advances in Code Summarization
Desai, U., Sridhara, G., & Tamilselvam, S. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion52605.2021.00141 · Cited by 3
Abdelaziz, I., Dolby, J., McCusker, J. P., & Srinivas, K. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i4.20363 · Cited by 5
Improving Code Autocompletion with Transfer Learning
Zhou, W. Z., Kim, S., Murali, V., & Aye, G. A. (2022) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip55303.2022.9793983 · Cited by 17
CLSEBERT: Contrastive Learning for Syntax Enhanced Code Pre-Trained Model.Preprint
Wang, X., Wang, Y., Zhou, P., Mi, F., Xiao, M., Wang, Y., Li, L., Liu, X., Wu, H., Liu, J., et al. (2021) · arXiv (Cornell University) · Cited by 3
Learning C to x86 Translation: An Experiment in Neural CompilationPreprint
Armengol-Estapé, J., & O’boyle, M. F. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.07639 · Cited by 16
A Stealthy Backdoor Attack for Code ModelsPreprint
qu, Y., Huang, S., Chen, X., Ju, X., Li, L., Wang, X., Yao, Y., & Xiao, Z. (2024) · Research Square · doi:10.21203/rs.3.rs-3969016/v1 · Cited by 3
R-U-SURE? Uncertainty-Aware Code Suggestions By Maximizing Utility Across Random User Intents
Johnson, D. D., Tarlow, D., & Walder, C. (2023) · arXiv · doi:10.48550/arxiv.2303.00732 · Cited by 3
Pbaeg: combine-vulnerabilities AEG to defeat protection mechanisms
Wang, Y., Zhai, Y., Li, Z., & Zhang, Y. (2025) · Cybersecurity · doi:10.1186/s42400-024-00322-9 · Cited by 2
KAPE: k NN-based Performance Testing for Deep Code Search
Guo, Y., Hu, Q., Xie, X., Cordy, M., Papadakis, M., & Traon, Y. L. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3624735 · Cited by 2
LenANet: A Length-Controllable Attention Network for Source Code Summarization
Chen, P., Wu, S., Chen, Z., Zhang, J., Zhang, X., & Feng, Z. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-8145-8_43 · Cited by 2
Self-citationAAHEG: Automatic Advanced Heap Exploit Generation Based on Abstract Syntax Tree
Wang, Y., Zhang, Y., & Li, Z. (2023) · Symmetry · doi:10.3390/sym15122197 · Cited by 2
Training a language model to learn the syntax of commands
Hussain, Z., Nurminen, J. K., & Ranta-aho, P. O. (2024) · Array · doi:10.1016/j.array.2024.100355 · Cited by 4
GenDetect: Generative Large Language Model Usage in Smart Contract Vulnerability Detection
Ince, P., Yu, J., Liu, J. K., Du, X., & Luo, X. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-2961-2_22 · Cited by 3
Multi-level Semantic Representation Model for Code Search.
Wen, D., Yang, L., Zhang, Y., Lin, Y. H., Xu, K., & Lin, H. (2020) · Cited by 2
Automatic Data-Driven Software Change Identification via Code Representation Learning
Heričko, T. (2023) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3593434.3593505 · Cited by 2
CCGRA: Smart Contract Code Comment Generation with Retrieval-enhanced Approach
Zhang, Z., Chen, S., Fan, G., Yang, G. W., & Feng, Z. (2023) · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · doi:10.18293/seke2023-090 · Cited by 2
Self-citationA Structural Transformer with Relative Positions in Trees for Code-to-Sequence Tasks
Villmow, J., Ulges, A., & Schwanecke, U. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533717 · Cited by 2
Cocktail: Leveraging Ensemble Learning for Optimized Model Serving in Public CloudPreprint
Gunasekaran, J. R., Mishra, C. S., Thinakaran, P., Kandemir, M., & Das, C. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05345 · Cited by 3
Learning to Generate Code Sketches.Preprint
Guo, D., Svyatkovskiy, A., Yin, J., Duan, N., Brockschmidt, M., & Allamanis, M. (2021) · arXiv (Cornell University) · Cited by 3
Self-citationChecking Patch Behaviour against Test SpecificationPreprint
Tian, H., Li, Y., Pian, W., Kaboré, A. K., Liu, K., Klein, J., & Bissyandé, T. F. (2021) · arXiv (Cornell University) · Cited by 2
Metadata
- Authors
- Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., Shou, L., Qin, B., Liu, T., Jiang, D., & Zhou, M.
- Affiliations
- Harbin Institute of Technology (CN)
- Sun Yat-sen University (CN)
- Microsoft (United States) (US)
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2002.08155
- OpenAlex
- W3008088841
- Cited by
- 243
- References
- 27
- Citations by year
- 2026: 9 · 2025: 41 · 2024: 51 · 2023: 28 · 2022: 23 · 2021: 72 · 2020: 18
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (2018YFB1005103, 61772156, 61632011)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- CodeBERT, natural language programming, transformer models, code generation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
