The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.
DeepSeek-Coder: When the Large Language Model Meets Programming - The Rise of Code Intelligence
Open accessAbstract
The rapid development of large language models has revolutionized code intelligence in software development. However, the predominance of closed-source models has restricted extensive research and development. To address this, we introduce the DeepSeek-Coder series, a range of open-source code models with sizes from 1.3B to 33B, trained from scratch on 2 trillion tokens. These models are pre-trained on a high-quality project-level code corpus and employ a fill-in-the-blank task with a 16K window to enhance code generation and infilling. Our extensive evaluations demonstrate that DeepSeek-Coder not only achieves state-of-the-art performance among open-source code models across multiple benchmarks but also surpasses existing closed-source models like Codex and GPT-3.5. Furthermore, DeepSeek-Coder models are under a permissive license that allows for both research and unrestricted commercial use.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Specifically, state-of-the-art models (Guo et al., 2024;Huang et al., 2023) show minimal performance differences between different algorithms.”
From Does Few-Shot Learning Help LLM Performance in Code Synthesis? (2024) · cites this paper as Guo et al., 2024
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The development of evaluations in tandem with language models (LMs) has substantially contributed to the rapid advancement of these models [29, 11, 7, 25, 81, 27, 72].”
From SciCode: A Research Coding Benchmark Curated by Scientists (2024) · cites this paper as [25]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Models released at different times, from DeepSeek-Coder in 2023 [10] to Qwen2.5Coder in 2024 [11] and Seed-Coder in 2025 [12], achieve similarly strong performance with only minor differences in average ES scores.”
From LongCodeZip: Compress Long Context for Code Language Models (2025) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“To exclude potential effects of data leakage, we used DeepSeek-Coder-6.7B [10], which was trained only on data available before March 2023, while all the benchmarks we evaluated were released after that date [1], [21], [13].”
From LongCodeZip: Compress Long Context for Code Language Models (2025) · cites this paper as [10]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“At the same time, different LLMs possess unique strengths and specialize in various tasks aspects.”
From Mixture-of-Agents Enhances Large Language Model Capabilities (2024) · cites this paper as Guo et al., 2024
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“LoRA is proposed to update partial parameters of LLMs by optimizing the low-rank decomposition of the attention module's matrices.”
From Exploring the Capabilities of LLMs for Code-Change-Related Tasks (2025) · cites this paper as [24]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 34
- L. B. Allal, R. Li, D. Kocetkov, C. Mou, C. Akiki, C. M. Ferrandis, N. Muennighoff, M. Mishra, A. Gu, M. Dey, et al. Santacoder: don't reach for the stars! arXiv preprint arXiv:2301.03988, 2023. · arXiv
- M. Bavarian, H. Jun, N. Tezak, J. Schulman, C. McLeavey, J. Tworek, and M. Chen. Efficient training of language models to fill in the middle. arXiv preprint arXiv:2207.14255, 2022. · arXiv
- F. Cassano, J. Gouwar, D. Nguyen, S. Nguyen, L. Phipps-Costin, D. Pinckney, M.-H. Yee, Y. Zi, C. J. Anderson, M. Q. Feldman, et al. Multipl-e: a scalable and polyglot approach to benchmarking neural code generation. IEEE Transactions on Software Engineering, 2023.
- M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. · arXiv
- S. Chen, S. Wong, L. Chen, and Y. Tian. Extending context window of large language models via positional interpolation. arXiv preprint arXiv:2306.15595, 2023. · arXiv
- P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018. · arXiv
- K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. · arXiv
- DeepSeek-AI. Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954, 2024. · arXiv
- Y. Ding, Z. Wang, W. U. Ahmad, H. Ding, M. Tan, N. Jain, M. K. Ramanathan, R. Nallapati, P. Bhatia, D. Roth, et al. Crosscodeeval: A diverse and multilingual benchmark for cross-file code completion. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023.
- Z. Du, Y. Qian, X. Liu, M. Ding, J. Qiu, Z. Yang, and J. Tang. Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 320-335, 2022.
- D. Fried, A. Aghajanyan, J. Lin, S. Wang, E. Wallace, F. Shi, R. Zhong, W.-t. Yih, L. Zettlemoyer, and M. Lewis. Incoder: A generative model for code infilling and synthesis. arXiv preprint arXiv:2204.05999, 2022. · arXiv
- L. Gao, A. Madaan, S. Zhou, U. Alon, P. Liu, Y. Yang, J. Callan, and G. Neubig. Pal: Program- aided language models. In International Conference on Machine Learning, pages 10764- 10799.
- PMLR, 2023.
- Z. Gou, Z. Shao, Y. Gong, Y. Yang, M. Huang, N. Duan, W. Chen, et al. Tora: A tool-integrated reasoning agent for mathematical problem solving. arXiv preprint arXiv:2309.17452, 2023. · arXiv
- D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. Measur- ing massive multitask language understanding. In International Conference on Learning Representations, 2020.
- D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. Mea- suring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. · arXiv
- D. Kocetkov, R. Li, L. Jia, C. Mou, Y. Jernite, M. Mitchell, C. M. Ferrandis, S. Hughes, T. Wolf, D. Bahdanau, et al. The stack: 3 tb of permissively licensed source code. Transactions on Machine Learning Research, 2022.
- V. A. Korthikanti, J. Casper, S. Lym, L. McAfee, M. Andersch, M. Shoeybi, and B. Catanzaro. Reducing activation recomputation in large transformer models. Proceedings of Machine Learning and Systems, 5, 2023.
- Y. Lai, C. Li, Y. Wang, T. Zhang, R. Zhong, L. Zettlemoyer, W.-t. Yih, D. Fried, S. Wang, and T. Yu. Ds-1000: A natural and reliable benchmark for data science code generation. In International Conference on Machine Learning, pages 18319-18345. PMLR, 2023.
- K. Lee, D. Ippolito, A. Nystrom, C. Zhang, D. Eck, C. Callison-Burch, and N. Carlini. Deduplicat- ing training data makes language models better. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8424-8445, 2022.
- R. Li, L. B. Allal, Y. Zi, N. Muennighoff, D. Kocetkov, C. Mou, M. Marone, C. Akiki, J. Li, J. Chim, et al. Starcoder: may the source be with you! arXiv preprint arXiv:2305.06161, 2023. · arXiv
- P. Lu, L. Qiu, K.-W. Chang, Y. N. Wu, S.-C. Zhu, T. Rajpurohit, P. Clark, and A. Kalyan. Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning. In The Eleventh International Conference on Learning Representations, 2022.
- S.-Y. Miao, C.-C. Liang, and K.-Y. Su. A diverse corpus for evaluating and developing english math word problem solvers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 975-984, 2020.
- D. Narayanan, A. Harlap, A. Phanishayee, V. Seshadri, N. R. Devanur, G. R. Ganger, P. B. Gibbons, and M. Zaharia. Pipedream: Generalized pipeline parallelism for dnn training. In Proceedings of the 27th ACM Symposium on Operating Systems Principles, pages 1-15, 2019.
- E. Nijkamp, B. Pang, H. Hayashi, L. Tu, H. Wang, Y. Zhou, S. Savarese, and C. Xiong. Codegen: An open large language model for code with multi-turn program synthesis. arXiv preprint arXiv:2203.13474, 2022. · arXiv
- A. Patel, S. Bhattamishra, and N. Goyal. Are nlp models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2080- 2094, 2021.
- S. Rajbhandari, J. Rasley, O. Ruwase, and Y. He. Zero: Memory optimizations toward training tril- lion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1-16. IEEE, 2020.
- B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y. Adi, J. Liu, T. Remez, J. Rapin, et al. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950, 2023. · arXiv
- K. Sakaguchi, R. L. Bras, C. Bhagavatula, and Y. Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99-106, 2021.
- M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou, , and J. Wei. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261, 2022. · arXiv
- H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023. · arXiv
- Y. Wang, W. Wang, S. Joty, and S. C. Hoi. Codet5: Identifier-aware unified pre-trained encoder- decoder models for code understanding and generation. arXiv preprint arXiv:2109.00859, 2021. · arXiv
- R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, and Y. Choi. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791-4800, 2019.
- Q. Zheng, X. Xia, X. Zou, Y. Dong, S. Wang, Y. Xue, L. Shen, Z. Wang, A. Wang, Y. Li, et al. Codegeex: A pre-trained model for code generation with multilingual benchmarking on humaneval-x. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 5673-5684, 2023.
Cited by
Sorted by citations
SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
Yang, J., Jimenez, C., Wettig, A., Lieret, K., Yao, S., Narasimhan, K., & Press, O. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-1601 · Cited by 264
Exploring DeepSeek: A Survey on Advances, Applications, Challenges and Future Directions
Deng, Z., Ma, W., Han, Q., Zhou, W., Zhu, X., Wen, S., & Xiang, Y. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2025.125498 · Cited by 184
Structured Chain-of-Thought Prompting for Code Generation
Li, J., Li, G., Li, Y., & Jin, Z. G. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3690635 · Cited by 308
Self-citationGenerative AI in cybersecurity: A comprehensive review of LLM applications and vulnerabilities
Ferrag, M. A., Alwahedi, F., Battah, A., Cherif, B., Mechri, A., Tihanyi, N., Bisztray, T., & Debbah, M. (2025) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2025.01.001 · Cited by 148
Liu, S., Fang, W., Lu, Y., Zhang, Q., Zhang, H., & Xie, Z. (2024) · IEEE LLM Aided Design Workshop (LAD) · doi:10.1109/lad62341.2024.10691788 · Cited by 109
Self-citationWang, F., Zhang, Z., Zhang, X., Wu, Z., Mo, T., Lu, Q., Wang, W., Li, R., Xu, J., Tang, X., et al. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3768165 · Cited by 106
Large language models in food science: Innovations, applications, and future
Ma, P., Tsai, S., He, Y., Jia, X., Zhen, D., Yu, N., Wang, Q., Ahuja, J. K. C., & Wei, C. (2024) · Trends in Food Science & Technology · doi:10.1016/j.tifs.2024.104488 · Cited by 99
UAVs meet LLMs: Overviews and perspectives towards agentic low-altitude mobility
Tian, Y., Lin, F., Li, Y., Zhang, T., Zhang, Q., Fu, X., Huang, J., Dai, X., Wang, Y., Tian, C., et al. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103158 · Cited by 139
Self-citationThe Dawn of Natural Language to SQL: Are We Fully Ready?
Li, B., Luo, Y., Chai, C., Li, G., & Tang, N. (2024) · Proceedings of the VLDB Endowment · doi:10.14778/3681954.3682003 · Cited by 179
RTLCoder: Fully Open-Source and Efficient LLM-Assisted RTL Code Generation Technique
Liu, S., Fang, W., Lu, Y., Wang, J., Zhang, Q., Zhang, H., & Xie, Z. (2024) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2024.3483089 · Cited by 77
Self-citationA comprehensive survey on integrating large language models with knowledge-based methods
Yang, W., Some, L., Bain, M., & Kang, B. H. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113503 · Cited by 108
Look Before You Leap: An Exploratory Study of Uncertainty Analysis for Large Language Models
Huang, Y., Song, J., Wang, Z., Zhao, S., Chen, H., Juefei-Xu, F., & Ma, L. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3519464 · Cited by 275
AIGC-empowered smart manufacturing: Prospects and challenges
Leng, J., Zheng, K., Li, R. C., Chen, C., Wang, B., Liu, Q., Chen, X., & Shen, W. (2025) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2025.103076 · Cited by 74
Rahman, A., Mahir, S. H., Tashrif, M. T. A., Karim, M. A., Aishi, A. A., Kundu, D., Debnath, T., Moududi, M. A. A., Eidmum, M. Z. A., Miah, A. S. M., et al. (2025) · Systems and Soft Computing · doi:10.1016/j.sasc.2025.200396 · Cited by 101
A Survey of Research in Large Language Models for Electronic Design Automation
Pan, J., Zhou, G.-S., Chang, C.-C., Jacobson, I., Hu, J., & Chen, Y. (2025) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3715324 · Cited by 88
DeepSeek vs. ChatGPT: prospects and challenges
Jin, I., Tangsrivimol, J. A., Darzidehkalani, E., Virk, H. U. H., Wang, Z., Egger, J., Hacking, S. M., Glicksberg, B. S., Strauß, M., & Krittanawong, C. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1576992 · Cited by 62
Chain-of-Thought in Neural Code Generation: From and for Lightweight Language Models
Yang, G., Zhou, Y., Chen, X., Zhang, X., Zhuo, T. Y., & Chen, T. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3440503 · Cited by 75
Large Language Model-Based Agents for Software Engineering: A Survey
Jun-wei, L., Wang, K., Chen, Y., Peng, X., Chen, Z., Zhang, L., & Lou, Y. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3796507 · Cited by 299
LLM4VV: Developing LLM-driven testsuite for compiler validation
Munley, C., Jarmusch, A., & Chandrasekaran, S. (2024) · Future Generation Computer Systems · doi:10.1016/j.future.2024.05.034 · Cited by 55
RepairLLaMA: Efficient Representations and Fine-Tuned Adapters for Program Repair
Silva, A., Fang, S., & Monperrus, M. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3581062 · Cited by 128
Wang, R., Gao, Z., Zhang, L., Yue, S., & Gao, Z. (2025) · Computer Science Review · doi:10.1016/j.cosrev.2025.100755 · Cited by 61
Qwen2.5-Coder Technical Report
Hui, B., Yang, J., Cui, Z., Yang, J., Liu, D., Zhang, L., Liu, T., Zhang, J., Yu, B., Lu, K., et al. (2024) · arXiv · doi:10.48550/arxiv.2409.12186 · Cited by 65
“5-Coder-7B-Base stands as the leading model among those over 6 billion parameters, matching the performance of the formidable 33 billion parameter model, DS-Coder-33B-Base.”
MethodInfluentialGeoCode-GPT: A large language model for geospatial code generation
Hou, S., Shen, Z., Zhao, A., Liang, J., Gui, Z., Guan, X., Li, R., & Wu, H. (2025) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2025.104456 · Cited by 53
Knowledge Transfer from High-Resource to Low-Resource Programming Languages for Code LLMs
Cassano, F., Gouwar, J., Lucchetti, F., Schlesinger, C., Freeman, A., Anderson, C. J., Feldman, M. Q., Greenberg, M., Jangda, A., & Guha, A. (2024) · Proceedings of the ACM on Programming Languages · doi:10.1145/3689735 · Cited by 88
Exploring the Capabilities of LLMs for Code-Change-Related Tasks
Fan, L., Liu, J., Liu, Z., Lo, D., Xia, X., & Li, S. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3709358 · Cited by 41
A Comprehensive Overview and Analysis of Large Language Models: Trends and Challenges
Mohammed, A., & Kora, R. (2025) · IEEE Access · doi:10.1109/access.2025.3573955 · Cited by 45
CRUXEVAL-X: A Benchmark for Multilingual Code Reasoning, Understanding and Execution
Xu, R., Cao, J., Lu, Y., Wen, M., Lin, H., Han, X., He, B., Cheung, S.-C., & Sun, L. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.1158 · Cited by 35
CellVoyager: AI CompBio agent generates new insights by autonomously analyzing biological data
Alber, S., Chen, B., Sun, E., Isakova, A., Wilk, A. J., & Zou, J. (2026) · Nature Methods · doi:10.1038/s41592-026-03029-6 · Cited by 60
SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
Yang, J., Jimenez, C. E., Wettig, A., Lieret, K., Yao, S., Narasimhan, K., & Press, O. (2024) · arXiv · doi:10.48550/arxiv.2405.15793 · Cited by 31
Project-specific code summarization with in-context learning
Yun, S. M., LIN, S.-J., Gu, X., & Shen, B. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112149 · Cited by 29
Fight Fire With Fire: How Much Can We Trust ChatGPT on Source Code-Related Tasks?
Yu, X., Liu, L., Hu, X., Keung, J., Liu, J., & Xia, X. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3492204 · Cited by 34
BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions
Zhuo, T. Y., Vu, M. C., Chim, J., Hu, H., Yu, W., Widyasari, R., Yusuf, I. N. B., Zhan, H., He, J., Paul, I., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.15877 · Cited by 29
Learning-based Widget Matching for Migrating GUI Test Cases
Zhang, Y., Zhang, W., Ran, D., Zhu, Q., Dou, C., Hao, D., Xie, T., & Zhang, L. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3623322 · Cited by 42
Self-citationOn the Effectiveness of LLM-as-a-Judge for Code Generation and Summarization
Crupi, G., Tufano, R., Velasco, A., Mastropaolo, A., Poshyvanyk, D., & Bavota, G. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3586082 · Cited by 55
RLCoder: Reinforcement Learning for Repository-Level Code Completion
Wang, Y., Wang, Y., Guo, D., Chen, J., Zhang, R., Ma, Y., & Zheng, Z. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00014 · Cited by 43
Self-citationChatAssert: LLM-Based Test Oracle Generation With External Tools Assistance
Hayet, I., Scott, A., & d’Amorim, M. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3519159 · Cited by 38
Mixture-of-Agents Enhances Large Language Model Capabilities
Wang, J., Wang, J., Athiwaratkun, B., Zhang, C., & Zou, J. (2024) · arXiv · doi:10.48550/arxiv.2406.04692 · Cited by 386
“For instance, some models excel at complex instruction following (Xu et al., 2023a) while others may be better suited for code generation (Roziere et al., 2023; Guo et al., 2024).”
BackgroundInfluentialStagedVulBERT: Multigranular Vulnerability Detection With a Novel Pretrained Code Model
Jiang, Y., Zhang, Y., Su, X., Treude, C., & Wang, T. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3493245 · Cited by 29
OmniSQL: Synthesizing High-Quality Text-to-SQL Data at Scale
Li, H., Wu, S., Zhang, X., Huang, X., Zhang, J., Jiang, F., Wang, S., Zhang, T., Chen, J., Shi, R., et al. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3749646.3749723 · Cited by 56
Qiskit Code Assistant: Training LLMs for generating Quantum Computing Code
Dupuis, N., Buratti, L., Vishwakarma, S., Forrat, A. V., Kremer, D., Faro, I., Puri, R., & Cruz-Benito, J. (2024) · IEEE LLM Aided Design Workshop (LAD) · doi:10.1109/lad62341.2024.10691762 · Cited by 42
Towards Understanding the Characteristics of Code Generation Errors Made by Large Language Models
Wang, Z., Zhou, Z., Song, D., Huang, Y., Chen, S., Ma, L., & Zhang, T. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00180 · Cited by 87
AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodology
Nguyen, M. H., Chau, T. P., Nguyen, P. X., & Bui, N. D. Q. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00026 · Cited by 68
Agentless: Demystifying LLM-based Software Engineering Agents
Xia, C. S., Deng, Y., Dunn, S., & Zhang, L. (2024) · arXiv · doi:10.48550/arxiv.2407.01489 · Cited by 24
“We recursively traverse the entire repository to obtain the structure, which will be used as input for the LLM.”
MethodFlexFL: Flexible and Effective Fault Localization With Open-Source Large Language Models
Xu, C., Liu, Z., Ren, X., Zhang, G., Liang, M., & Lo, D. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3553363 · Cited by 42
Xie, C., Gao, H., Huang, Y., Xue, Z., Xu, C. B., & Dai, K. (2025) · Earthquake research advances · doi:10.1016/j.eqrea.2025.100378 · Cited by 24
WebCode2M: A Real-World Dataset for Code Generation from Webpage Designs
Gui, Y., Li, Z., Wan, Y., Shi, Y., Zhang, H., Chen, B., Su, Y., Chen, D., Wu, S., Zhou, X., et al. (2025) · Proceedings of the ACM on Web Conference 2025 · doi:10.1145/3696410.3714889 · Cited by 17
Code Summarization Beyond Function Level
Makharev, V., & Ivanov, V. V. (2025) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1109/llm4code66737.2025.00024 · Cited by 25
Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering
Si, C., Zhang, Y., Li, R., Yang, Z., Liu, R., & Yang, D. (2025) · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2025.naacl-long.199 · Cited by 17
“…such as Codex (Chen et al., 2021), Star-Coder (Li et al., 2023), InCoder (Fried et al., 2022), CodeLlama (Rozière et al., 2023), and DeepSeek-Coder (Guo et al., 2024), enable a wave of programming support applications such as automatic code completion and infilling, and allowing users to chat with…”
Self-citationCOFFE: A Code Efficiency Benchmark for Code Generation
Peng, Y., Wan, J., Li, Y., & Ren, X. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3715727 · Cited by 15
Self-citationCan large language models generate geospatial code?
Hou, S., Shen, Z., Liang, J., Jiao, H., Zhao, A., Qing, Y., Peng, D., Gui, Z., Guan, X., Xiang, L., et al. (2025) · Geo-spatial Information Science · doi:10.1080/10095020.2025.2535523 · Cited by 19
Type-Constrained Code Generation with Language Models
Mündler-Sasahara, N., He, J.-X., Wang, H., Sen, K., Song, D., & Vechev, M. T. (2025) · Proc. ACM Program. Lang. · doi:10.1145/3729274 · Cited by 77
Fixing Large Language Models' Specification Misunderstanding for Better Code Generation
Tian, Z., Chen, J.-J., & Zhang, X.-Y. (2025) · International Conference on Software Engineering · doi:10.1109/icse55347.2025.00108
Assessing and improving syntactic adversarial robustness of pre-trained models for code translation
Yang, G., Zhou, Y., Zhang, X., Chen, X., Han, T., & Chen, T. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107699 · Cited by 12
Is This You, LLM? Recognizing AI-written Programs with Multilingual Code Stylometry
Gurioli, A., Gabbrielli, M., & Zacchiroli, S. (2025) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner64311.2025.00044 · Cited by 18
LLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-Based Code Completion
Wang, C., Huang, K., Zhang, J., Feng, Y., Zhang, L., Liu, Y., & Peng, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00245 · Cited by 17
Intention is All you Need: Refining your Code from your Intention
Guo, Q., Xie, X., Liu, S., Hu, M., Li, X., & Bu, L. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00191 · Cited by 17
A knowledge-informed dynamic correlation modeling framework for lane-level traffic flow prediction
Jiang, R., Wang, S., Ma, W., Zhang, Y., Fan, P., & Jia, D. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103327 · Cited by 12
An empirical study on capability of Large Language Models in understanding code semantics
Nguyen, T.-T., Vu, T. T., Vo, H. D., & Nguyen, S. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107780 · Cited by 21
Evaluating quantized Large Language Models for code generation on low-resource language benchmarks
Nyamsuren, E. (2025) · Journal of Computer Languages · doi:10.1016/j.cola.2025.101351 · Cited by 18
Rango: Adaptive Retrieval-Augmented Proving for Automated Software Verification
Thompson, K., Saavedra, N., Carrott, P., Fisher, K. D., Sanchez-Stern, A., Brun, Y., Ferreira, J. F., Lerner, S., & First, E. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00161 · Cited by 14
LLMSecCode: Evaluating Large Language Models for Secure Coding
Rydén, A., Näslund, E. B., Schiller, E. M., & Almgren, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-76934-4_7 · Cited by 9
Zhang, C., You, Y., Wang, N., Park, J., & Zhang, L. (2025) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2025.3575272 · Cited by 13
Hewedy, D. W., & Abdelhady, A. S. (2025) · doi:10.1109/iceeng64546.2025.11031311 · Cited by 8
A Transformer-Based Approach for Smart Invocation of Automatic Code Completion
de Moor, A., van Deursen, A., & Izadi, M. (2024) · Proceedings of the 1st ACM International Conference on AI-Powered Software · doi:10.1145/3664646.3664760 · Cited by 7
Tao, W., Su, X., Ke, Y., Han, Y., Zheng, Y., & Wei, H. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113341 · Cited by 7
Enhancing Code Generation for Low-Resource Languages: No Silver Bullet
Giagnorio, A., Martin-Lopez, A., & Bavota, G. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00058 · Cited by 10
Unseen Horizons: Unveiling the Real Capability of LLM Code Generation Beyond the Familiar
Zhang, Y., Xie, Y., Lit, S., Liu, P., Wang, C., Jia, Z., Huang, X., Song, J., Luo, C., Zheng, Z., et al. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00082 · Cited by 7
AutoPrep: Natural Language Question-Aware Data Preparation with a Multi-Agent Framework
Fan, M., Fan, J., Tang, N., Cao, L., Li, G., & Du, X. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3748191.3748211 · Cited by 25
An exploration of prompting LLMs to generate energy-efficient code
Cappendijk, T., de Reus, P., & Oprescu, A. (2025) · IEEE/ACM International Workshop on Green and Sustainable Software (GREENS) · doi:10.1109/greens66463.2025.00010 · Cited by 7
Exploring the Potential of LLMs for Code Deobfuscation
Beste, D., Menguy, G., Hajipour, H., Fritz, M., Ciná, A. E., Bardin, S., Holz, T., Eisenhofer, T., & Schönherr, L. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-97620-9_15 · Cited by 7
Peng, M., Xie, Y., Guo, X., Yao, R., Yang, H., & Ma, J. (2026) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2026.105694 · Cited by 21
ShellBox: Adversarially Enhanced LLM-Interactive Honeypot Framework
Yang, G., Sun, Z., & Wang, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3598779 · Cited by 9
Lv, J., Si, J., Gao, D., & Bao, J. (2025) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2025.10.005 · Cited by 6
Sen, Ö., Pohl, C., Hacker, I., Stroot, M., & Ulbig, A. (2024) · IEEE International Conference on Cyber Security and Resilience (CSR) · doi:10.1109/csr61664.2024.10679448 · Cited by 6
Anchor Attention, Small Cache: Code Generation With Large Language Models
Zhang, X., Zhou, Y., Yang, G., Gall, H. C., & Chen, T. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3570680 · Cited by 6
Sentiment-Enhanced Cyberbullying Detection Models on Social Media Platforms
Philipo, A. G., Ding, J., Sarwatt, D. S., Mohamed, J. A., Yusufu, A. S., Daneshmand, M., & Ning, H. (2025) · ACM Transactions on the Web · doi:10.1145/3766075 · Cited by 8
Dumitran, A. M., Badea, A. C., & Muscalu, S.-G. (2024) · International Conference on Innovations in Intelligent Systems and Applications (INISTA) · doi:10.1109/inista62901.2024.10683837 · Cited by 5
Evaluating Large Language Models for Code Generation: A Comparative Study on Python, Java, and Swift
Wan, B. (2025) · Applied and Computational Engineering · doi:10.54254/2755-2721/2025.tj22242 · Cited by 5
Evaluating the Generalizability of LLMs in Automated Program Repair
Li, F., Jiang, J., Sun, J., & Zhang, H. (2025) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier66352.2025.00024 · Cited by 11
Generating P4 Dataplanes Using LLMs
Dumitru, M.-V., Bădoiu, V.-A., Gherghescu, A. M., & Raiciu, C. (2024) · IEEE International Conference on High Performance Switching and Routing (HPSR) · doi:10.1109/hpsr62440.2024.10635926 · Cited by 5
LLM4VV: Exploring LLM-as-a-Judge for Validation and Verification Testsuites
Sollenberger, Z., Patel, J., Munley, C., Jarmusch, A., & Chandrasekaran, S. (2024) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/scw63240.2024.00238 · Cited by 5
aiXcoder-7B: A Lightweight and Effective Large Language Model for Code CompletionPreprint
Jiang, S., Li, J., Zong, H., Liu, H., Zhu, H., Hu, S., Li, E. M., Ding, J., Han, Y., Ning, W., et al. (2024) · Qeios · doi:10.32388/atahd0 · Cited by 6
LongCodeZip: Compress Long Context for Code Language Models
Shi, Y., Qian, Y., Zhang, H., Shen, B., & Gu, X. (2025) · 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00020 · Cited by 7
DocSpider: a dataset of cross-domain natural language querying for MongoDB
Özer, A. G., Çekinel, R. F., Toroslu, İ. H., & Karagöz, P. (2025) · Natural language processing. · doi:10.1017/nlp.2024.63 · Cited by 6
Experimental Evaluation of Parameter-Efficient Fine-Tuning for Software Engineering Tasks
Zou, W., Shen, Z., Li, Q., Ge, J., Li, C., Chen, X., Shen, X., Huang, L., & Luo, B. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3722107 · Cited by 16
Improving LLM-Based Verilog Code Generation with Data Augmentation and RL
Min, K., Park, S., Park, H., Cho, J., & Kang, S. (2025) · Design, Automation, and Test in Europe · doi:10.23919/date64628.2025.10992897 · Cited by 5
Huang, F., Xu, Z., Min, X., & Song, S. (2025) · Displays · doi:10.1016/j.displa.2025.103103 · Cited by 5
Enhancing LLM Performance on Hardware Design Generation Task via Reinforcement Learning
Zhao, Y., Fu, W., Li, S., Hu, Y., Guo, X., & Jin, Y. (2025) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas56072.2025.11043832 · Cited by 4
Can test cases generated by large language models facilitate automated program repair?
Zhang, C., Wang, H., Xu, C., Liu, J., Liu, K., & Liu, Z. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10802-w · Cited by 4
Fu, Y., Li, B., Li, L., Zhang, W., & Xie, T. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728947 · Cited by 7
Self-citationCoding agents: A comprehensive survey of automated bug fixing systems and benchmarks
Puvvadi, M., Arava, S. K., Santoria, A., Chennupati, S. S. P., & Puvvadi, H. V. (2025) · 2025 IEEE 14th International Conference on Communication Systems and Network Technologies (CSNT) · doi:10.1109/csnt64827.2025.10968728 · Cited by 4
Liu, P., Chen, Z., Li, Y., & Wong, W. E. (2025) · International Symposium on System Security, Safety, and Reliability (ISSSR) · doi:10.1109/isssr65654.2025.00047 · Cited by 4
Self-citationSciCode: A Research Coding Benchmark Curated by Scientists
Tian, M., Gao, L., Zhang, S. D., Chen, X., Fan, C., Guo, X., Haas, R., Ji, P., Krongchon, K., Li, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.13168 · Cited by 15
“The development of evaluations in tandem with language models (LMs) has substantially contributed to the rapid advancement of these models [29, 11, 7, 25, 81, 27, 72].”
BackgroundMethodSelf-citationOn the Impacts of Contexts on Repository-Level Code Generation
Hai, N. L., Nguyen, D. M., & Bui, N. D. Q. (2024) · arXiv · doi:10.48550/arxiv.2406.11927 · Cited by 3
Zhou, T., Yang, Y., Yang, C., Xiao, L., Liu, X., & Hu, M. (2025) · IEEE International Conference on Big Data Security on Cloud, IEEE International Conference on High Performance and Smart Computing, and IEEE International Conference on Intelligent Data and Security (BigDataSecurity/HPSC/IDS) · doi:10.1109/hpsc66065.2025.00016 · Cited by 3
ChatHPC: Building the Foundations for a Productive and Trustworthy AI-Assisted HPC Ecosystem
Valero‐Lara, P., Young, A. R., Vetter, J. S., Jin, Z., Pophale, S., Monil, M. A. H., Teranishi, K., & Godoy, W. F. (2025) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3712285.3759787 · Cited by 3
The Art of Repair: Optimizing Iterative Program Repair with Instruction-Tuned Models
Ruiz, F. V., Hort, M., & Moonen, L. (2025) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3756681.3756966 · Cited by 13
Lin, Z., Huang, Q., Zhong, J., Xie, J., Wang, Y., & Weiying, H. (2026) · Social Network Analysis and Mining · doi:10.1007/s13278-025-01546-4 · Cited by 4
Harnessing Large Language Models for Curated Code Reviews
Sghaier, O. B., Weyssow, M., & Sahraoui, H. A. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00039 · Cited by 5
Knowledge-Enhanced Program Repair for Data Science Code
Ouyang, S., Zhang, J. M., Sun, Z., & Meroño-Peñuela, A. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00246 · Cited by 6
Metadata
- Authors
- Guo, D., Zhu, Q., Yang, D., Xie, Z., Dong, K., Zhang, W., Chen, G., Bi, X., Wu, Y., Li, Y. K., Luo, F., Xiong, Y., & Liang, W.
- Affiliations
- Peking University (CN)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2401.14196
- OpenAlex
- W4391272793
- Cited by
- 180
- References
- 0
- Citations by year
- 2026: 39 · 2025: 109 · 2024: 32
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Keywords
- DeepSeek-Coder, code generation, code intelligence, permissive licenses
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
