Competition-level code generation with AlphaCode
Open accessThe DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Abstract
Programming is a powerful and ubiquitous problem-solving tool. Systems that can assist programmers or even generate programs themselves could make programming more productive and accessible. Recent transformer-based neural network models show impressive code generation abilities yet still perform poorly on more complex tasks requiring problem-solving skills, such as competitive programming problems. Here, we introduce AlphaCode, a system for code generation that achieved an average ranking in the top 54.3% in simulated evaluations on recent programming competitions on the Codeforces platform. AlphaCode solves problems by generating millions of diverse programs using specially trained transformer-based networks and then filtering and clustering those programs to a maximum of just 10 submissions. This result marks the first time an artificial intelligence system has performed competitively in programming competitions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“AlphaCode, Self-Debugging and CodeRL are effective in fixing less-complex program bugs, but they rely upon ground truth test cases that invalidate pass@1 eligibility, and do not use self-reflection to bridge the gap between error identification and implementation improvement.”
From Reflexion: Language Agents with Verbal Reinforcement Learning (2023) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Large language models are increasingly capable of complex programming tasks and are already being used to accelerate AI research, from generating training data to serving as programming tools [1, 2, 3, 4, 5, 6].”
From RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts (2024) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Generative AI, such as OpenAI’s ChatGPT, has rapidly emerged as a disruptive technology capable of achieving human-level performance on a broad range of tasks (1–5).”
From Generative AI without guardrails can harm learning: Evidence from high school mathematics (2025) · cites this paper as [3]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Li et al. [27] showed that the accuracy of AlphaCode increases from close to 0% zero-shot to over 15% with 1,000 retries and over 30% with a million retries (accuracy is measured by how often one of the top 10 answers generated by the model is correct).”
From AI Agents That Matter (2024) · cites this paper as Li et al. [27]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Evaluation covered 200 CodeContests tasks [15] across three seeds and up to three feedback rounds, with online tests supplying feedback and hidden tests measuring final success.”
From RSI-Anything: Unlocking Recursive Self-Improvement with World Models (2026) · cites this paper as [15]
Experimental — accuracy not yet measured
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Self-citationDecoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×
“Following Kaplan et al. (2020), we adjusted the amount of training for other model sizes such that larger models are trained more and smaller models are trained less to optimize the use of compute.”
MethodGrandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455 · cited 2×
“We believe that sampling metadata leads to a more diverse set of model samples, a strategy similar to that used by Vinyals et al. (2019), and allows our model to take advantage of the relative strengths across the metadata distribution.”
MethodSelf-citationResnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum, E., Silver, J., Silverman, B., et al. (2009) · Communications of the ACM · doi:10.1145/1592761.1592779 · Cited by 3,333
Kudo, T., & Richardson, J. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/d18-2012 · Cited by 2,869
Research works printed but not linked: 6
- A. Church, Application of recursive arithmetic to the problem of circuit synthesis.. J. Symb. Log. 28, 289–290 (1963).
- Y. Nandwani D. Jindal Mausam P. Singla ?oNeural learning of one-of-many solutions for combinatorial problems in structured output spaces ?? International Conference on Learning Representations (ICLR 2021) 3 to 7 May 2021.
- A. Holtzman J. Buys L. Du M. Forbes Y. Choi ?oThe curious case of neural text degeneration ?? Proceedings of the 8th International Conference on Learning Representations (ICLR 2020) 26 April to 1 May 2020.
- J. Ganitkevitch B. Van Durme C. Callison-Burch ?oPPDB: The paraphrase database ?? Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Atlanta Georgia 9 to 15 June 2013 (Association for Computational Linguistics 2013) pp. 758–764.
- A. Solar-Lezama ?oProgram synthesis by sketching ?? thesis University of California Berkeley (2008).
- J. Cai R. Shin D. Song ?oMaking neural programming architectures generalize via recursion ?? Proceedings of the 5th International Conference on Learning Representations (ICLR) Toulon France 24 to 26 April 2017.
Cited by
Items 801–900 of 1,104
iCLP: Large Language Model Reasoning with Implicit Cognition Latent Planning
Chen, S., & Niu, D. (2025) · arXiv · doi:10.48550/arxiv.2512.24014
Multi-LCB: Extending LiveCodeBench to Multiple Programming Languages
Ivanova, M., Zadorozhny, P., Levichev, R., Petrov, I., Pavel, A., Lopatin, I., Kutalev, A., & Babaev, D. (2026) · arXiv · doi:10.48550/arxiv.2606.20517 · Cited by 1
“…language models (LLMs) have recently demonstrated impressive capabilities in code-related tasks (Ridnik et al., 2024; Lozhkov et al., 2024; Roziere et al., 2023; Li et al., 2022; Nijkamp et al., 2022), powering applications such as AI-assisted programming, automated debugging, and code translation.”
Measuring The Impact Of Programming Language Distribution
Orlanski, G., Xiao, K., Garcia, X., Hui, J., Howland, J., Malmaud, J., Austin, J., Singh, R., & Catasta, M. (2023) · arXiv · doi:10.48550/arxiv.2302.01973 · Cited by 21
Jiang, J., Shen, J., Kim, S., Yoo, K. M., Kim, J., & Kim, S. (2026) · arXiv · doi:10.48550/arxiv.2603.05863 · Cited by 8
“…these advancements, standard “System 1” approaches which generate solutions in a single forward pass face an inherent ceiling when tackling complex, multi-step algorithmic problems (Li et al., 2022; Chen et al., 2024; Bairi et al., 2023; Wang et al., 2024; Park et al., 2025; Zhong et al., 2024).”
What Makes In-Context Examples Effective for Code Generation?
Li, D., Chen, S., Cao, J., & Cheung, S.-C. (2026) · Proceedings of the ACM on Software Engineering · doi:10.1145/3832275
Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?
Li, W., Lin, Y., Xia, M., & Jin, C. (2025) · arXiv · doi:10.48550/arxiv.2502.00674 · Cited by 62
“…with automatic verifiers available, such as math (Hendrycks et al., 2021) and code (Chen et al., 2021), simply sampling LLMs multiple times can significantly improve the pass@k metric and hence boost the success rate of solving the tasks (Roziere et al., 2023; Li et al., 2022; Brown et al., 2024).”
Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?
Xia, C. S., Wang, Z., Yang, Y., Wei, Y., & Zhang, L. (2025) · arXiv · doi:10.48550/arxiv.2511.13646 · Cited by 2
“1 Introduction Large Language Models (LLMs) have rapidly progressed from simple code autocompletion [7, 11, 21] to interactive agents capable of navigating repositories, running tests, and submitting patches end-to-end [40, 33, 45, 36, 9, 22].”
Learning to Code with Context: A Study-Based Approach
Borghoff, U. M., Minas, M., & Schopp, J. (2026) · Software · doi:10.3390/software5020027
“Models such as DeepMind’s A LPHA C ODE [27], Google’s P A LM [28], Microsoft’s C ODE BERT [29], and OpenAI’s C HAT GPT and C ODEX [30] generate syntactically correct and functionally meaningful code from natural-language prompts.”
Are Performance-Optimization Benchmarks Reliably Measuring Coding Agents?
Chen, Z., Sun, Z., Shi, Y., Lo, D., & Jiang, L. (2026) · arXiv · doi:10.48550/arxiv.2607.01211 · Cited by 9
“Earlier benchmarks focus on function-level edits [4], standalone programs [5], [6], and function-level and file-level code generation tasks [7]–[9].”
AutoHarness: improving LLM agents by automatically synthesizing a code harness
Lou, X., Lázaro-Gredilla, M., Dedieu, A., Wendelken, C., Lehrach, W., & Murphy, K. P. (2026) · arXiv · doi:10.48550/arxiv.2603.03329
“In the domain of program synthesis, methods like AlphaCode (Li et al., 2022) utilize large-scale sampling and filtering, whereas AlphaEvolve (Novikov et al., 2025) applies an evolutionary algorithm to entire codebases using an LLM as a mutation function.”
Liu, X., & Bohlin, L. (2014)
Gu, J., & Gall, H. C. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3613076 · Cited by 2
Select-Them-Repair-Them: Automatically Optimize Pseudocode-to-Code Data
Yu, Q., & Gu, N. (2023) · IEEE International Conference on Big Data Analytics (ICBDA) · doi:10.1109/icbda57405.2023.10104712
AI learns to write sorting software on its own
Solar-Lezama, A. (2023) · Nature · doi:10.1038/d41586-023-01812-5
Future Trend of Network Security
Liu, J., Bai, L., Jiang, C., & Zhang, W. (2023) · Springer eBooks · doi:10.1007/978-981-99-1125-7_6 · Cited by 2
Zum Einsatz von KI als Methode und Werkzeug in den Medienwissenschaften
Sudmann, A. (2023) · Springer eBooks · doi:10.1007/978-3-658-36629-2_16-1
PREFACE. FOTONIKA-LV since 2010
Úbelis, A. (2023) · doi:10.22364/iesg2123.01
MCMC Correction of Score-Based Diffusion Models for Model Composition
Sjöberg, A., Lindqvist, J., Önnheim, M., Jirstrand, M., & Svensson, L. (2026) · Entropy · doi:10.3390/e28030351 · Cited by 7
A STUDY ON CONSUMER PERCEPTION TOWARDS MINERAL WATER
(2023) · International Journal of Progressive Research in Engineering Management and Science · doi:10.58257/ijprems31932
LGEQRE: Learning Guided Enumerative Synthesis for Query Reverse EngineeringPreprint
Wang, H., Dou, Q., Tang, H., Pan, H., & Zhang, S. (2023) · Research Square · doi:10.21203/rs.3.rs-3320857/v1
Transformers-Based Automated PHP Code Generator
Tomer, Y., Sharma, R., & Pandey, R. (2023) · Lecture notes in networks and systems · doi:10.1007/978-981-99-2602-2_44
An Integrated Search System for Korea Weather Data
Jo, J., Ki, D., Yoon, S.-Y., & Seo, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-industry.71
Gu, J., Nan, Z., Peng, Z., Shen, X., & Xu, D. (2023) · doi:10.18653/v1/2023.pandl-1.7
Intervention-Based Alignment of Code Search with Execution Feedback
Han, H., Kim, M., Hwang, S.-W., Duan, N., & Lu, S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.148 · Cited by 3
TSTR: Target Similarity Tuning Meets the Real World
Khatry, A., Gulwani, S., Gupta, P., Le, V., Singh, M. P., Singha, A., & Verbruggen, G. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.687
Analyzing Code Text Strings for Code Evaluation
Wang, T., & Chen, Z. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386406 · Cited by 3
Retracted: Analyse and Evaluate Quixbugs with Open AI Codex and Powering Next Generation Application
Tyagi, D., Tanwar, S., Mittal, N., & Badotra, S. (2023) · International Conference on Contemporary Computing and Informatics (IC3I) · doi:10.1109/ic3i59117.2023.10397753
RetractedJayachandran, D., Maldikar, P. S., & Blum, J. J. (2024) · Zone 1 Conference of the American Society for Engineering Education (ASEE Zone 1) · doi:10.18260/1-2--45088 · Cited by 1
Huang, Z., Dutta, S., & Misailovíc, S. (2024) · International Journal on Software Tools for Technology Transfer · doi:10.1007/s10009-024-00737-2 · Cited by 2
SYNC: A Structurally Guided Hard Negative Curricula for Generalizable Neural Code Search
Naik, A., Das, S., Vedurada, J., & Aditya, S. (2023) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2023.ijcnlp-main.7 · Cited by 1
Artificial Intelligence und Quantum Computing
Wirtz, B. W. (2024) · Springer eBooks · doi:10.1007/978-3-658-41467-2_8
A machine learning approach towards SKILL code autocompletion
Dehaerne, E., Dey, B., Meert, W., & Ridder, V. D. (2024) · DTCO and Computational Patterning · doi:10.1117/12.3008988 · Cited by 1
Neural code generation for robust automatic program repair
Wang, W. (2024) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/173910 · Cited by 2
A Survey on Code Pre-Trained Models for Neural Program RepairPreprint
Zhan, S., Wang, X., Wei, D., & Cao, X. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4831589
INTEGRATING CHATGPT IN PRIMARY EDUCATION: OPPORTUNITIES AND CONSIDERATIONS
Ospanbekova, M., Turikpenova, S. Z., Izmagambetova, R., Yertayeva, P. K., & Temirkhanova, K. (2024) · BULLETIN OF THE NATIONAL ACADEMY OF SCIENCES OF THE REPUBLIC OF KAZAKHSTAN ( THE BULLETIN) · doi:10.32014/2024.2518-1467.722
Advancements in Machine Learning and Deep Learning
Darwish, D. A. M. (2024) · Advances in geospatial technologies book series · doi:10.4018/979-8-3693-2913-9.ch006 · Cited by 2
Tan, J. C. M., & Motani, M. (2024) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai59869.2024.00149 · Cited by 27
Future of Competitive Programming
Laaksonen, A. (2024) · Undergraduate topics in computer science · doi:10.1007/978-3-031-61794-2_18
Automatic Code Scraping Based Test Case Generation
Chinnaswamy, A., & Sabarish, B. A. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-60935-0_42
Unveiling the Decision-Making Process in Reinforcement Learning with Genetic Programming
Eberhardinger, M., Rupp, F., Maucher, J., & Maghsudi, S. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-7181-3_28 · Cited by 2
Domainko, W. F. (2024) · Springer eBooks · doi:10.1007/978-3-662-69339-1_8
Exploring Visual and Multimodal Interaction in NCL Authoring
Borges, P. V., Moraes, D. D. S., dos Santos, J. A. F., Muchaluat-Saade, D. C., & Colcher, S. (2024) · Brazilian Symposium on Multimedia and the Web · doi:10.5753/webmedia.2024.243157 · Cited by 2
黄海辉 (2024) · 教育学刊 · doi:10.62836/jer.v2n4.0156
de Diego, Á. G., & Wotawa, F. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-80889-0_5
Application of Compact Large Language Models to the Named Entity Recognition Problem
Lorkiewicz, W. A., Hojda, M., & Popek, G. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-79086-7_2
Enhancing Code LLMs with Reinforcement Learning in Code Generation: A SurveyPreprint
He, Y., Shi, T., Li, Y., & Chen, Q. (2025) · Qeios · doi:10.32388/8g8tb2
Self-citationAGENTS: A Historical Perspective 1948-2024Preprint
Alonso, M. N. I. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5067907
Tex2Py-45K: A Parallel Corpus Dataset for Bidirectional Conversion Between LaTeX and Python
Li, L., Duan, M., & Wang, Y. (2024) · doi:10.1145/3709026.3709044
MP: motion program synthesis with machine learning interpretability and knowledge graph analogy
Cai, C. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00495-8
The attitudes and perceptions of Chinese Literature students in Mandarin learning using ChatGPT
Go, Y., & Khosasih, A. (2025) · Linguistics Initiative · doi:10.53696/27753719.51226
Forecasting Graph-Based Time-Dependent Data with Graph Sequence Attention
Li, Y., Wang, D., & Moura, J. M. F. (2025) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3721435 · Cited by 1
Self-citationHinglishEval: Evaluating the Effectiveness of Code-Generation Models on Hinglish Prompts
Pawagi, M., Gupta, A., Rolla, S. R., & Saha, K. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-84391-4_2 · Cited by 1
Implementation of Validation Rules in the Process of Automating Graphical User Interface Generation
Mihajlović, M. T., & Antović, I. D. (2025) · doi:10.1109/it64745.2025.10930253
Vuković, M., Vujović, V., Milinković, S., Štaka, Z., & Čavoški, S. (2025) · International Symposium INFOTEH-JAHORINA (INFOTEH) · doi:10.1109/infoteh64129.2025.10959264 · Cited by 1
Review and Mapping of Search-Based Approaches for Program Synthesis
Saber, T., & Tao, N. (2025) · Information · doi:10.3390/info16050401 · Cited by 2
Hort, M., & Moonen, L. (2025) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst62969.2025.10988963 · Cited by 1
AlphaLoss: Evolving Portfolio Optimization Loss FunctionsPreprint
Yüksel, K. A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5263279 · Cited by 1
From NL2SQL to NL2Code: Exploring A New Paradigm for Natural Language to Programming Code Generation
Tang, L., Hu, Z., & Liu, Y. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-5373-7_90 · Cited by 1
Anh, D. H., Tran, V., & Nguyen, L.-M. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-7071-0_12
The Game Boy Learning Environment
Fazzari, E., Romano, D., Falchi, F., & Stefanini, C. (2025) · IEEE Transactions on Games · doi:10.1109/tg.2025.3575527 · Cited by 2
Wen, Y., Cui, Z., Liu, Y., Zhang, Z., Zhou, J., & Tang, L. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07487-1 · Cited by 1
Evaluating Language Models for Computer Graphics Code Completion
Kels, J., Dahou, A. H., & Mathiak, B. (2025) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1109/llm4code66737.2025.00017
Identifying AI-generated and Human-written Answers in Sinhala Using a Deep Learning Approach
Ranathunga, R. A. D. K., Rupasingha, R. A. H. M., & Kumara, B. T. G. S. (2025) · International Research Conference on Smart Computing and Systems Engineering (SCSE) · doi:10.1109/scse65633.2025.11031028 · Cited by 1
SnipGen: A Mining Repository Framework for Evaluating LLMs for Code
Rodríguez-Cárdenas, D., Velasco, A., & Poshyvanyk, D. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00084 · Cited by 3
Betin, V. N., Ivashchenko, V. A., & Suprun, A. P. (2025) · Automatic Documentation and Mathematical Linguistics · doi:10.3103/s0005105525700542
Algorithmic Inversion: A Learnable Algorithm Representation for Code Generation
Shi, Z., Wu, F., Zeng, W., Kong, Y., Shen, S., & Wu, Y. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00054
Rouabhia, D., & Hadjadj, I. (2025) · Research Square · doi:10.21203/rs.3.rs-6793290/v1 · Cited by 9
LLM-Based MaSE, A Software Development Framework for Developing Multi-agent Systems
Hajjarzadeh, S., Abad, Z. S. H., & Far, B. H. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-8889-0_37
Benchmarking Large Language Models for Data Pipeline Code Generation and ExecutionPreprint
Rucco, C., Saad, M., Martina, T., & Longo, A. (2025) · Research Square · doi:10.21203/rs.3.rs-6786102/v1
Testing Android Third Party Libraries with LLMs to Detect Incompatible APIs
Mahmud, T., Duan, B., Che, M., Ngu, A. H. H., & Yang, G. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00039
Xie, Z. (2025) · Applied and Computational Engineering · doi:10.54254/2755-2721/2025.po24517
PTC: Prefix Tuning CodeT5 for High-Quality Secure Network Measurement Script Generation
Wang, Y., & Han, X. (2025) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn64029.2025.00024 · Cited by 1
Zhijie, Z., Deng, Y., Qi, H., & Wang, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5351223
DPEfficR: a data and parameter efficient approach for training neural API recommendation model
Yu, H., han, X., Chen, S. H., Feng, X., Sun, G., & Yang, W. P. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00530-8 · Cited by 2
Privacy-Preserving Training of Support Vector Machines via Secure Multiparty Computation
Jaramillo, D. C., Madrigal, H. D. V., Escudero, D. E., & Morales, F. A. (2025) · ACM Transactions on Privacy and Security · doi:10.1145/3749373
AutoBackend: Agent-Driven Framework for Intelligent Backend Development
Lin, R., Hu, B., & Zhang, Z. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-9986-5_35
Towards AI-Augmented Co-Compilation for Smart IoT Systems Through Domain-Specific LLMs
Ciccozzi, F. (2025) · International Conference on Smart and Sustainable Technologies (SpliTech) · doi:10.23919/splitech65624.2025.11091755
Yonathan, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5367078 · Cited by 1
Taibi, D., & Fulantelli, G. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-94002-6_3 · Cited by 2
Spec2Doc2RTL: RTL Generation from Specification with Natural Language Representation
Chen, Z., Lin, Z., Chen, X., Liu, Z., Liu, C., Qiao, Y., Feng, Y., Zuo, J., Song, Y., & Yang, F. (2025) · International Symposium of Electronics Design Automation (ISEDA) · doi:10.1109/iseda65950.2025.11101140 · Cited by 1
Self-citationThink, Prune, Train: Can Small Models Teach Themselves to Reason?
Costello, C., Guo, S., Goldie, A., & Mirhoseini, A. (2025) · IEEE International Conference on LLM-Aided Design (ICLAD) · doi:10.1109/iclad65226.2025.00012
Repository-Level Code Generation Method Enhanced by Context-Dependent Graph Retrieval
Zhang, H., Li, A., Kuai, Z., Wei, Z., Kuang, L., & Xia, Y. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-93257-1_3
Benchmarking AI-Facilitated UI Test-Script Generation: A Reproducible Evaluation Framework
Poth, A., Rrjolli, O., & Wang, H. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-04288-0_9
Exploiting Contexts of LLM-based Code-Completion
Noppel, M., Rubel, K., & Wressnegger, C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-02813-6_26
LevDetectCode: Zero-Shot Detection of AI-Generated Code Using Levenshtein Distance
Fu, J., Shen, G., Huang, Z., Yu, Y., & Zhao, Y. D. (2025) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194025500524
Enhancing MATLAB Code Generation with Large Language Models
Liu, X., & Jin, K. (2025) · IEEE International Conference on Image Processing and Computer Applications (ICIPCA) · doi:10.1109/icipca65645.2025.11138499
Deploy Efficient Large Language Model Distributed Inference Pipeline for Heterogeneous GPUs
Zhang, J., Hou, J., Zhang, B., & Li, X. (2025) · International Workshop on Quality of Service · doi:10.1109/iwqos65803.2025.11143255
Shao, Y., Zhang, K., Zhou, Y., Chen, Z., Yang, Y., & Peng, L. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.112193
Weesommai, P., Nuannimnoi, S., & Huang, C. (2025) · International Conference on Computer Communication and the Internet (ICCCI) · doi:10.1109/iccci65070.2025.11158389
A Little Help Goes a Long Way: Tutoring LLMs in Solving Competitive Programming Through Hints
Zhang, Y., Dong, W., Liu, J., Wang, S., Wang, D., Ma, T., Li, Y., & Yang, K. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3614204 · Cited by 4
Self-citationMulti-scale Swarm of Large Language Models for Python Code Generation
Tan, Y., Duan, Q., Sun, L., & Shi, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-0982-9_14
Closing the Benchmark Gap for Tiered Memory
Shashidhara, R., Peter, S., Hare, S., & Keeton, K. (2025) · Workshop on Disruptive Memory Systems (DIMES) · doi:10.1145/3764862.3768177
Transforming natural language into code: advancing automated code synthesis for software development
shboul, B. A., Qaqour, A., Bani-Salameh, H., & Almakadmeh, K. (2025) · Cluster Computing · doi:10.1007/s10586-025-05688-0
Tell-Tale Watermarks for Explanatory Reasoning in Synthetic Media Forensics
Chang, C., & Echizen, I. (2026) · IEEE Access · doi:10.1109/access.2026.3660000 · Cited by 1
BLADE: Brainstorming LLMs as algorithm designer through evolution for online subset selection
Qin, Z., Wang, C., Guo, J., Qin, H., Shen, P., & Weijia Jia, B. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.115890 · Cited by 1
Darmawan, R. P., & Wibowo, S. A. (2025) · IEEE International Symposium on Future Telecommunication Technologies (SOFTT) · doi:10.1109/softt67007.2025.11213109
Automated Update of Android Deprecated API Usages With Large Language Models
Mahmud, T., Duan, B., Che, M., Yasmin, A., Ngu, A. H. H., & Yang, G. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3627897 · Cited by 9
Metadata
- Authors
- Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., Hubert, T., Choy, P., de Masson d’Autume, C., Babuschkin, I., Chen, X., Huang, P.-S., Welbl, J., Gowal, S., Cherepanov, A., Molloy, J., Mankowitz, D. J., Sutherland Robson, E., Kohli, P., de Freitas, N., Kavukcuoglu, K., & Vinyals, O.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- DeepMind, London, UK.
- Corresponding author
- Li, Yujia
- Choi, David
- Chung, Junyoung
- Kushman, Nate
- Schrittwieser, Julian
- Leblond, Rémi
- Eccles, Tom
- Keeling, James
- Gimeno, Felix
- Choy, Peter
- Vinyals, Oriol
- ORCID
- Venue
- Science
- Published
- 2022-12-09
- Volume
- 378
- Issue
- 6624
- Pages
- 1092-1097
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W4226287673
- Cited by
- 1,043
- References
- 34
- Citations by year
- 2026: 269 · 2025: 319 · 2024: 237 · 2023: 186 · 2022: 30
- FWCI
- 191.10
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- automatic code generation, competitive programming, programming competition, Codeforces, transformer language models, AI-assisted programming
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.