The DOI matches its DataCite record; 76 of the 92 references it cites were found in a registry or an index, and 1 are not checked yet.
Code Llama: Open Foundation Models for Code
Open accessAbstract
We release Code Llama, a family of large language models for code based on Llama 2 providing state-of-the-art performance among open models, infilling capabilities, support for large input contexts, and zero-shot instruction following ability for programming tasks. We provide multiple flavors to cover a wide range of applications: foundation models (Code Llama), Python specializations (Code Llama - Python), and instruction-following models (Code Llama - Instruct) with 7B, 13B, 34B and 70B parameters each. All models are trained on sequences of 16k tokens and show improvements on inputs with up to 100k tokens. 7B, 13B and 70B Code Llama and Code Llama - Instruct variants support infilling based on surrounding content. Code Llama reaches state-of-the-art performance among open models on several code benchmarks, with scores of up to 67% and 65% on HumanEval and MBPP, respectively. Notably, Code Llama - Python 7B outperforms Llama 2 70B on HumanEval and MBPP, and all our models outperform every other publicly available model on MultiPL-E. We release Code Llama under a permissive license that allows for both research and commercial use.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Given a new task, current LLMs consume and learn from the task data 'as-is' via finetuning or in-context learning [9, 10, 11, 12].”
From Self-Adapting Language Models (2025) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Code-based LLMs (e.g., CodeLlama(Roziere etal. 2023) and StarCoder(Li etal. 2023d; Lozhkov et al. 2024)) are particularly well suited to address this demand.”
From When LLMs meet cybersecurity: a systematic literature review (2025) · cites this paper as Roziere et al. 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“At the time of writing, only the CodeLlama models (Rozi` ere et al., 2023) are able to handle the very long contexts necessary.”
From SWE-bench: Can Language Models Resolve Real-World GitHub Issues? (2023) · cites this paper as Rozière et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Code Llama family are based upon Llama 2. It is pre-trained on a 500B token corpus with both code and natural language texts using the infilling objective.”
From Exploring the Capabilities of LLMs for Code-Change-Related Tasks (2025) · cites this paper as [67]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“which is a 7B decoder-only model trained on code synthesis.”
From Does Few-Shot Learning Help LLM Performance in Code Synthesis? (2024) · cites this paper as Roziere et al., 2023
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs have achieved remarkable proficiency at generating code in mainstream programming languages such as Python and JavaScript, yet this proficiency does not transfer to logic programming.”
From From Reasoning to Code: GRPO Optimization for Underrepresented Languages (2026) · cites this paper as Rozière et al. (2023)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“CodeLLaMA-34B [42] is an aligned version of LLaMA-234B, which is fine-tuned with 500B tokens of code data.”
From Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation (2024) · cites this paper as [42]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“At the same time, different LLMs possess unique strengths and specialize in various tasks aspects.”
From Mixture-of-Agents Enhances Large Language Model Capabilities (2024) · cites this paper as Roziere et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“There have been many previous works that have confirmed that LLMs do have good code comprehension capabilities (He et al. 2024; Luo et al. 2024; Roziere et al. 2023; Li et al. 2023d).”
From When LLMs meet cybersecurity: a systematic literature review (2025) · cites this paper as Roziere et al. 2023
The cited source says what the claim attributes to it.
In the citing paper
“Moreover, while numerous LMs have been implemented with a primary focus on code generation, most tend to underperform the best generalist LMs (Li et al., 2022; Le et al., 2022; Roziere et al., 2023; Austin et al., 2021; Xu et al., 2022).”
From Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation (2023) · cites this paper as Roziere et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Utilizing SWE-bench-train, we release two fine-tuned models, SWE-Llama 7 b and 13 b, based on the CodeLlama (Rozi` ere et al., 2023) model.”
From SWE-bench: Can Language Models Resolve Real-World GitHub Issues? (2023) · cites this paper as Rozière et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Prior studies showed that within the same LLM family, larger models are associated with better performance [59, 67, 73].”
From Exploring the Capabilities of LLMs for Code-Change-Related Tasks (2025) · cites this paper as [67]
The cited source says what the claim attributes to it.
In the citing paper
“The main experiments are performed on CODELLAMA (Roziere et al., 2023).”
From Does Few-Shot Learning Help LLM Performance in Code Synthesis? (2024) · cites this paper as Roziere et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Furthermore CODELLAMA is more specialized in coding tasks thus yielding slightly lower PERPLEX-ITY (TARGET) score compared to MISTRAL despite their comparable sizes.”
From Does Few-Shot Learning Help LLM Performance in Code Synthesis? (2024) · cites this paper as Roziere et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Among the LLMs tested, CODELLAMA (Roziere et al., 2023) benefits the most from good test case selection, hence we focus this work around CODELLAMA.”
From Does Few-Shot Learning Help LLM Performance in Code Synthesis? (2024) · cites this paper as Roziere et al., 2023
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Moreover, both of these tools aim to maximise the acceptance rate of the shown completions”
From A Transformer-Based Approach for Smart Invocation of Automatic Code Completion (2024) · cites this paper as [32]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Specifically, optimising only acceptance rate results in worse completions [32, 49].”
From A Transformer-Based Approach for Smart Invocation of Automatic Code Completion (2024) · cites this paper as [32]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Consistent with earlier research [32, 49], we acknowledge that these proxy metrics might not fully capture the usability of the interaction without potentially compromising it in some other way.”
From A Transformer-Based Approach for Smart Invocation of Automatic Code Completion (2024) · cites this paper as [32]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“and contains most of the significant features found in previous work [32].”
From A Transformer-Based Approach for Smart Invocation of Automatic Code Completion (2024) · cites this paper as [32]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×
“Effective handling of long sequences is a major topic of research in transformer-based language modeling (Vaswani et al., 2017).”
BackgroundBLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 3×
“( b) Difference in BLEU scores for single line code completion of long context models compared to their respective base models before fine-tuning.”
MethodInfluentialRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · cited 1×
“To measure the toxicity of the generated outputs from each of the LLMs, we employ the default ToxiGen classifier, tuned on RoBERTa (Liu et al., 2019).”
MethodExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“…2023; Nguyen et al., 2023), many tasks related to code understanding or synthesis have been addressed since the early 2020s with NLP models adapted for code (Raffel et al., 2020; Feng et al., 2020; Guo et al., 2021; Wang et al., 2021; Ahmad et al., 2021), also see the survey by Xu & Zhu (2022).”
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087 · cited 2×
“The VADER produces sentiment scores between -1 and 1, where a positive (negative) score indicates a positive (negative) sentiment towards the population mentioned in the prompt.”
MethodLLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · cited 11×
“ChatGPT and GPT-4 (OpenAI, 2023), PaLM (Chowdhery et al., 2022) and Chinchilla (Hoffmann et al., 2022) are closed source, while BLOOM (Scao et al., 2022), OPT (Zhang et al., 2022b), and the seminal work of Llama are public (Touvron et al., 2023a).”
BackgroundMethodInfluentialSelf-citationKudo, T., & Richardson, J. (2018) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-2012 · Cited by 4,454 · cited 1×
“� 500B 20B 20B”
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · cited 1×
“Long sequences in LLMs. Scaling Transformers and LLMs to long input sequences has attracted much recent interest (Dai et al., 2019; Beltagy et al., 2020; Yu et al., 2023; Ding et al., 2023).”
BackgroundTraining Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117 · cited 3×
“Collecting supervised data from human annotators or training from human feedback (Ouyang et al., 2022) is expensive for coding tasks as it requires input from professional developers.”
BackgroundMethodModel Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Research works printed but not linked: 6
- Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernandez Abrego, Junwhan Ahn, Jacob Austin, Paul Barham, Jan Botha, James Bradbury, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, Colin Cherry, Christopher A. Choquette-Choo, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, Mostafa Dehghani, Sunipa Dev, Jacob Devlin, Mark Díaz, Nan Du, Ethan Dyer, Vlad Feinberg, Fangxiaoyu Feng, Vlad Fienber, Markus Freitag, Xavier Garcia, Sebastian Gehrmann, Lucas Gonzalez, Guy Gur-Ari, Steven Hand, Hadi Hashemi, Le Hou, Joshua Howland, Andrea Hu, Jeffrey Hui, Jeremy Hurwitz, Michael Isard, Abe Ittycheriah, Matthew Jagielski, Wenhao Jia, Kathleen Kenealy, Maxim Krikun, Sneha Kudugunta, Chang Lan, Katherine Lee, Benjamin Lee, Eric Li, Music Li, Wei Li, YaGuang Li, Jian Li, Hyeontaek Lim, Hanzhao Lin, Zhongtao Liu, Frederick Liu, Marcello Maggioni, Aroma Mahendru, Joshua Maynez, Vedant Misra, Maysam Moussalem, Zachary Nado, John Nham, Eric Ni, Andrew Nystrom, Alicia Parrish, Marie Pellat, Martin Polacek, Alex Polozov, Reiner Pope, Siyuan Qiao, Emily Reif, Bryan Richter, Parker Riley, Alex Castro Ros, Aurko Roy, Brennan Saeta, Rajkumar Samuel, Renee Shelby, Ambrose Slone, Daniel Smilkov, David R. So, Daniel Sohn, Simon Tokumine, Dasha Valter, Vijay Vasudevan, Kiran Vodrahalli, Xuezhi Wang, Pidong Wang, Zirui Wang, Tao Wang, John Wieting, Yuhuai Wu, Kelvin Xu, Yunhan Xu, Linting Xue, Pengcheng Yin, Jiahui Yu, Qiao Zhang, Steven Zheng, Ce Zheng, Weikang Zhou, Denny Zhou, Slav Petrov, and Yonghui Wu. PaLM 2 Technical Report. arXiv:abs/2305.10403, 2023.
- Jaehun Jung, Peter West, Liwei Jiang, Faeze Brahman, Ximing Lu, Jillian Fisher, Taylor Sorensen, and Yejin Choi. Impossible distillation: From low-quality model to high-quality dataset & model for summarization and paraphrasing. arXiv:abs/2305.16635, 2023.
- Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019.
- Han Peng, Ge Li, Wenhan Wang, Yunfei Zhao, and Zhi Jin. Integrating tree path in transformer for code representation. In NeurIPS, pp. 9343-9354, 2021.
- Vighnesh Leonardo Shiv and Chris Quirk. Novel positional encodings to enable tree-based transformers. In NeurIPS, pp. 12058-12068, 2019.
- Michele Tufano, Dawn Drain, Alexey Svyatkovskiy, Shao Kun Deng, and Neel Sundaresan. Unit test case generation with transformers. arXiv:abs/2009.05617, 2020.
Cited by
Sorted by citations
Items 501–600 of 609
The approach development of data extraction from lambda terms
Deineha, O., Donets, V., & Zholtkevych, G. (2024) · Eastern-European Journal of Enterprise Technologies · doi:10.15587/1729-4061.2024.298991 · Cited by 2
“The most advanced broad-based models are the Code Llama models [16], which include three possible model sizes for parameters 7B, 13B, and 34B.”
BackgroundCodeSift: An LLM-Based Reference-Less Framework for Automatic Code Validation
Aggarwal, P., Chatterjee, O., Dai, T., Mohapatra, P., Paulovicks, B. D., Blancett, B., & Magalhaes, A. D. (2024) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud62652.2024.00052
“We analyze the performance of our framework on the code generated by four prominent models: Starcoder [27], Codellama 34B [28], ChatGPT [29], and Mistral 7B [30].”
MethodAn Academic Information Collection System based on Large Language Model
Lang, Q., Lyu, S., Wang, H., & Zhang, R. (2024) · International Conference on Electronic Communication and Artificial Intelligence (ICECAI) · doi:10.1109/icecai62591.2024.10675124
“Large Language Models (LLMs) have demonstrated their exceptional ability across various applications [3,10,11] as LLMs have been trained on vast amount of textual data [4-6].”
BackgroundEvaluating the Performance of Large Language Models for Design Validation
Rahman, A., & Fey, G. (2024) · IEEE International System-on-Chip Conference (SOCC) · doi:10.1109/socc62300.2024.10737717
“5 [18], ChatGPT-4 [19], Gemini Pro [20] and, CodeLlama 13B Instruct [21] are chosen.”
Code Generation Based on Quality-Aware Reordering
Yuan, S., & Yu, L. (2024) · International Conference on Intelligent Computing and Signal Processing (ICSP) · doi:10.1109/icsp62122.2024.10743331
Privacy-Decision Assisting Techniques
Kqiku, L. (2024) · eDiss (Georg-August-Universität Göttingen) · doi:10.53846/goediss-10852
Tan, Z., Chen, Q., Zhu, W., Huang, Y., & Liang, C. (2024) · Neural computing & applications (Print) · doi:10.1007/s00521-024-10669-2
Hacking, C., de Boer, B., Verbeek, H., Hamers, J. P. H., & Aarts, S. (2024) · JMIR Human Factors · doi:10.2196/57911
Towards generalizable neural program understanding and synthesis
Li, Z. (2024) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/181413
Liu, X., Yin, D., Zhang, C., Zhao, D., & Feng, Y. (2025) · Computational Linguistics · doi:10.1162/coli_a_00548 · Cited by 4
“Nowadays, decoder-only architectures have become the most prevalent choice (Roziere et al. 2023; Guo et al. 2024).”
InfluentialEnhancing Python Code Embeddings: Fusion of Code2vec with Large Language Models
Ngo, L. H., & Rivalan, J. (2025) · Journal of Engineering Research and Sciences · doi:10.55708/js0401001
Enhancing Code LLMs with Reinforcement Learning in Code Generation: A SurveyPreprint
He, Y., Shi, T., Li, Y., & Chen, Q. (2025) · Qeios · doi:10.32388/8g8tb2
Tex2Py-45K: A Parallel Corpus Dataset for Bidirectional Conversion Between LaTeX and Python
Li, L., Duan, M., & Wang, Y. (2024) · doi:10.1145/3709026.3709044
“Code-Llama-Python-34B [28] is initialized from LLaMA2 weights and trained on 500B tokens from the Code Llama dataset, specialized on 100B tokens using a Python-heavy dataset, long context fine-tuned on 20B tokens.”
MP: motion program synthesis with machine learning interpretability and knowledge graph analogy
Cai, C. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00495-8
“In other words, developers can jointly use language-based program synthesisers (Li et al. 2022; Rozière et al. 2023) to deal with requirements in natural languages and use MP to deal with requirements in graphics.”
VisQ2SQL: Towards SQL-Driven Data Visualization via LLMs-Grounded Preference Learning
Shi, S., Ren, T., & Hu, J. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10888696 · Cited by 1
“Seq2Vis [8] 96.97% 80.70% 75.52% 72.00% ncNet [14] 98.62% 83.13% 84.84% 77.80% RGVisNet [15] 95 an encoder-decoder model for text-to-text transformations, and CodeLlama [29] which specializes in code generation tasks.”
InfluentialFine-Tuning Large Language Models for Code-Style Analysis: The Significance of Dataset Size
Holovko, A., & Alieksieiev, V. (2025) · International Journal of Computing · doi:10.47839/ijc.24.1.3885
“Although LLMs are primarily known for their text-generation capabilities, several studies have explored their applications in classification tasks [11, 12], automated code reviews [13] and code understanding [14].”
Mathai, T. S., Kim, B., Stroie, O. M., & Summers, R. M. (2025) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01478-7
“…following LLMs: Vicuna [31] 7-Billion (Vicuna-7B), Vicuna 13-Billion (Vicuna-13B), Llama-2 [32] 7-Billion Chat (Llama-2-7B-chat), Llama–2 13-Billion Chat (Llama–2-13B-chat), CodeLlama [33] 7-Billion (CodeLlama-7B), Xwin [34] 7-Billion (Xwin-LM-7B), and TenyxChat [35, 36] 7-Billion (TenyxChat-7B).”
Guetari, R., Kaabi, R. S., Hamzaoui, H., & Kraïem, N. (2025) · Research Square · doi:10.21203/rs.3.rs-6353409/v1
Wicaksono, A. T., & Jati, H. (2025) · Journal of Information Engineering and Technology (JIETY) · doi:10.21831/jiety.v2i2.931
Integrating GenAI in Advancing Game Product Management and Development
Pilaniwala, P. (2024) · International Conference on Parallel, Distributed and Grid Computing (PDGC) · doi:10.1109/pdgc64653.2024.10984420
“AI enables computers to think and learn, enhancing their ability to perform tasks such as understanding and generating code [42].”
LLM4SPS – Evaluierung von Large Language Models für die Generierung von SPS-Code
Pfeiffer, J., Kimmel, R., Wortmann, A., & Zhang, J. (2025) · Zeitschrift für wirtschaftlichen Fabrikbetrieb · doi:10.1515/zwf-2024-0115
LLMs for Virtualized Networking Infrastructures: An Industrial Report
Pannocchi, L., Lahiri, S., Fichera, S., Artale, A., & Cucinotta, T. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-90200-0_13
Watermarks for Large Language Models
Chen, P., & Liu, S. (2025) · Springer eBooks · doi:10.1007/978-3-031-76770-8_15
Reductive Analysis with Compiler-Guided Large Language Models for Input-Centric Code Optimizations
Wang, X., Hui, X., Liao, C., & Shen, X. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3729282
“LLMs have been trained on source code including CodeBERT [10], CodeT5 [38], CodeGen[25] and CODELLAMA[29].”
CoUpJava: A Dataset of Code Upgrade Histories in Open-Source Java Repositories
Jiang, K., Jin, B., & Nie, P. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00075
“Large Language Models (LLMs) trained on programming languages and natural languages [7], [16], [22] have demonstrated prosperous promise in reshaping software development across various realms of applications, such as code editing [19], [31], [32], API migration [9], [11], [15], refactoring [25],…”
Coft: Making Large Language Models Better Zero-Shot Learners for Code Generation
Li, W., Qian, Y., Gao, K., Chen, H., Wang, X., Tong, Y., Li, L., Wu, Y., & Zhao, C. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00059 · Cited by 1
“Although the [19], CodeGen [20], StarCoder [10], SantaCoder [21], CodeT5+ [22], CodeLLaMA [14], and DeepSeek Coder [16].”
Advancing Large Language Models in Code Generation: Usaco Benchmark and Bug Mitigation Insights
Trentini, J., Liu, V., Peng, Y., & Zong, Z. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00057 · Cited by 1
“Multiple open-source and closed source LLMs now exist for the purpose of code generation, of which we choose to evaluate Llama 3 [36], CodeGemma [37], Codestral [38], Phi3 (3B and 14B) [39], Mistral [40], Deepseek-Coder [41], DBRX, [42] CodeLlama (7B and 70B) [43], and GPT-4o [44].”
Structure-Guided and Semantics-Enhanced Collaborative Code Generation
Yang, D., Zhang, X., & Cao, Y. (2025) · International Conference on Advanced Electronic Materials, Computers and Software Engineering (AEMCSE) · doi:10.1109/aemcse65292.2025.11042693 · Cited by 1
Moghadam, I. H., Lijzenga, O., & Zaytsev, V. (2025) · ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences (GPCE) · doi:10.1145/3742876.3742881
LLM-Based MaSE, A Software Development Framework for Developing Multi-agent Systems
Hajjarzadeh, S., Abad, Z. S. H., & Far, B. H. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-8889-0_37
A Comprehensive Study of Bug Characteristics on Foundation Language Models
Han, J., Wang, G., Liu, J., Bao, L., Hu, X., Wei, J., & Deng, S. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00037
“CodeLLaMa [7], Medical LLM of ChatDoctor [8], Legal LLM of ChatLaw [9], and API management LLM of Gorilla [10].”
Self-citationSkill over Scale: The Case for Medium, Domain-Specific Models for SE
Mukherjee, M., & Hellendoorn, V. J. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00031 · Cited by 7
“We also considered the open-source CodeLLaMA-Instruct [38] and StackL-LaMA [39] models, which are pre-trained on code and natural language data, respectively, to leverage their domain-specific knowledge and capabilities.”
HyRACC: A Hybrid Retrieval-Augmented Framework for More Efficient Code Completion
Li, C., Shang, J., Feng, Y., & Luo, B. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00013
Su, F.-Y., Ngo, G.-H., Phan, B., & Chiang, J.-H. (2025) · Database · doi:10.1093/database/baaf025 · Cited by 1
Semantic Analysis of ChatGPT’s Behavior in Contextual Code Correction for Python
Alhalangy, A. (2025) · International Journal of Computational and Experimental Science and Engineering · doi:10.22399/ijcesen.3419
LLM Agents for Verifiable Question Generation and Grading
Levine, J., West, M., & Silva, M. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-99264-3_22
BRIDGING DESIGN AND DEVELOPMENT: BUILDING A GENERATIVE AI PLATFORM FOR AUTOMATED CODE GENERATION
Chennamsetty, C. S. (2025) · INTERNATIONAL JOURNAL OF COMPUTER ENGINEERING & TECHNOLOGY · doi:10.34218/ijcet_16_02_038
Evaluating LLMs for Effective Object Search in Video Data
David, R., Lee, J., Constantine, D., Rahman, M. M., & Mahmoud, Q. H. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-031-97992-7_63
Key-Augmented Neural Triggers for Knowledge SharingPreprint
Wolf, A., Palma, M. A., Rani, P., & Gall, H. C. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5376750
Spec2Doc2RTL: RTL Generation from Specification with Natural Language Representation
Chen, Z., Lin, Z., Chen, X., Liu, Z., Liu, C., Qiao, Y., Feng, Y., Zuo, J., Song, Y., & Yang, F. (2025) · International Symposium of Electronics Design Automation (ISEDA) · doi:10.1109/iseda65950.2025.11101140
Zhan, S., Wang, X., Wei, D., Cao, X., & Chen, B. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112590
EFIM: Efficient Serving of LLMs for Infilling Tasks with Improved KV Cache Reuse
Guo, T., Dong, H., Leng, Y., Liu, F., Lin, C., Xiao, N., & Zhang, X. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-99857-7_24
On Evaluating LLM Integration into Robotic Architectures
Sarathy, V., Fawn, M., McWilliams, M., Scheutz, M. J., & Oosterveld, B. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3754340
“We used pre-trained Llama-2 [19, 20] 3B and 7B models publicly available and open-sourced via the Huggingface ecosystem.”
Ahmed, S., & Sadeque, F. (2025) · doi:10.2196/preprints.82545
RiverEcho: Real-Time Interactive Digital System for Ancient Yellow River Culture
Wang, H., Guo, Y., Li, Z., Yue, T., Wang, Y., Zhang, E., Lin, R., Gao, F., Wang, S., & Ma, S. (2025) · IEEE International Conference on Multimedia and Expo workshops · doi:10.1109/icmew68306.2025.11152181 · Cited by 1
“Through instruction tuning and RAG, contemporary LLMs have achieved not only impressive general intelligence but also notable expertise in specialized domains [11]–[14].”
Lei, Z., Li, Y., Li, Z., Huang, X., Yu, D., Xue, N., & Chen, Y. (2025) · Internet of Things · doi:10.1016/j.iot.2025.101767
FAIRJupyter4AI: A Corpus of Computational Notebooks for AI
Mietchen, D., & Samuel, S. (2025) · Research Ideas and Outcomes · doi:10.3897/rio.11.e171656
API-Repo: API-centric Repository-level Code Completion
Li, Z., Li, C., Niu, C., Yan, Y., Ge, J., & Luo, B. (2025) · Asia-Pacific Symposium on Internetware · doi:10.1145/3755881.3755913
Li, Y., Peng, J., Yang, L., Li, Y., Shi, F., Xue, P., & Xu, C. (2025) · Communications in computer and information science · doi:10.1007/978-981-95-4088-4_7
Repository-level code completion with adaptive segmentation and fused retrieval
Shen, Y., Qin, P., Zhao, K., Zhang, F., Zhang, F., & Li, G. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10762-7
“Subsequently, LLMs tailored by researchers specifically for programming languages, such as StarCoder (Li et al. 2023 ), CodeLlama (Roziere et al. 2023 ), and DeepSeek-Coder (Guo et al. 2024 ) have shown robust performance in various tasks such as code-text summariza - tion (Zhu et al. 2024) code repair (Jin et al. 202…”
CodeRankEval: Benchmarking and Analyzing LLM Performance for Code Ranking
Chen, L.-G., Xiao, Z., Xu, Y.-J., An, R.-C., Wang, X., Li, Y.-N., Li, Y.-H., Wang, Y.-D., Zeng, Z.-R., Gao, Q., et al. (2025) · Journal of Computer Science and Technology · doi:10.1007/s11390-025-5514-9
Development of a Red-Teaming Dataset to Defend Large Language Models Against Attacks
Alekseevskaia, I., Arkhipenko, K. V., & Турдаков, Д. Ю. (2025) · Programming and Computer Software · doi:10.1134/s0361768825700343
“Nowadays, artificial intelligence (AI) systems are rapidly developed, being widely used in various fields, both as assistants (e.g., ChatGPT [1]) and for more practical tasks (e.g., Code Llama [27] for code generation).”
Testing like Mad Libs: Fuzzing SMT Solvers with Historical Unusual Inputs Empowered by LLMs
Sun, M., Yang, Y., Jia, H., Wu, J., Li, Q., Xie, Z., Wen, M., & Zhou, Y. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3796526
“We randomly sample 1,000 seeds from the bug-triggering formulas, as well as the mutants generated based on them, and examine the differences in s-expressions between the mutants and their corresponding seeds.”
InfluentialTheBotCompany: Self-Organizing Multi-agent Systems for Continuous Software Development
Lyu, W., Xiao, Y., Zhang, Y., & Sun, Y. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834358 · Cited by 3
AdvGen-X: Transferability driven adversarial example generation for pre-trained models of code
Liu, X., Liu, X., Sun, X., Bo, L., Li, B., Wu, X., Cao, S., & Hu, Y. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10864-w
“Extending AdvGen-X to recent and extremely large Code LLMs presents an exciting direction that aligns with our framework’s design (e.g., CodeLlama Rozière et al., 2023, StarCoder Li et al., 2023).”
Yuan, L., Liu, S., Wang, Y., Zhu, S., Wang, T., Mao, T., & Shao, S. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10838-y
“Subsequent models, such as Code LLaMA (Rozière et al. 2024), have further advanced code generation performance.”
Struggling at the Start: Structural Causes of Decoding Difficulty in Code Generation
Zhu, Y., Li, J., Jiang, H., Li, S., Zhang, J., & Li, A. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774905.3795840
“We investigates the differences in decoding behaviors between code and natural language generation using three LLMs: DeepSeek-Coder-7B[3], Qwen-Coder-7B[5], and CodeLlama-13B[12].”
Kátai, Z., & Iclănzan, D. (2026) · Acta Universitatis Sapientiae Informatica · doi:10.1007/s44427-026-00033-8
“Thislineofworkunderscoredtheviabilityofaccessibleresearch anddeploymentpathsbeyondclosedmodels[19,20].”
Asgari, A., Olsthoorn, M., & Panichella, A. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832236
“The evaluated models vary substantially in size, ranging from hundreds of millions of CodeLLaMA–7B [58] Tesoro [42] Binary 1,255 0.4375 CodeT5+ [65] Multi-label 1,255 0.4821 GraphCodeBERT [27] Multi-label 1,255 0.7004 Magicoder–6.7B [67 ] Binary 1,255 0.7171 Phi–2 [36] Multi-label 1,255 0.5729…”
InfluentialBeyond Language Boundaries: Uncovering Programming Language Families for Code Language Models
Yun, S. M., Gu, X., Huang, J. Y., & Shen, B. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797138
“While existing approaches to training code LLMs largely rely on the simple integration of multilingual code data [2, 7, 20, 33], they rarely explore the deeper linguistic relationships among programming languages or how these relationships could be leveraged to enhance model training.”
PlayCoder: Making LLM-Generated GUI Code Playable
Peng, Z., Tao, W., Yin, X., Ying, C., Luo, Y., & Guo, Y. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808097 · Cited by 10
Influence-Aware Bayesian-Inspired Token Reweighting for Improved Code Generation
Zhu, Y., Li, G., Mei, H., Jin, Z. G., Li, J., Zheng, Q., & Zhang, J. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808200
Sivapiran, S., & Uddin, G. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808123 · Cited by 1
“5-Coder-7B [20], CodeLlama-7b [43], deepseek-coder-1.”
DuCodeMark: Dual-Purpose Code Dataset Watermarking via Style-Aware Watermark-Poison Design
Chen, Y., Xiao, Y., Fang, C., Chen, Z., & Xu, B. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797076
In Line with Context: Repository-Level Code Generation via Context Inlining
Hu, C., Zeng, W., Shi, Y., Shen, B., & Gu, X. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797094 · Cited by 2
“Like most existing repository-level code generation benchmarks [8, 20, 23, 26, 27, 61, 63], our empirical evaluation is conducted on Python repositories, which may limit the generalizability of our findings to other programming languages or different types of software projects.”
Compiling Code LLMs into Lightweight Executables
Shi, J., He, J., Yang, Z., Yang, C., Klymenko, M. V., Hoang, T., Xu, X., Xing, Z., & Lo, D. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808196
CertiCoder: Towards MISRA-Compliant C Code Generation with LLMs
Gou, M., Yao, Z., Ma, H., Zhang, E.-D., Zhou, J., & He, F. (2026) · Proc. ACM Softw. Eng. · doi:10.1145/3797120
Zhang, L., Chen, C., Yan, D., Xu, H., Cao, M., Song, J., Cai, Z., & Guo, Y. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797073
Zhang, J., Li, T., Zhang, X., Hu, Q., & Xu, Y. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3786582.3786842 · Cited by 3
Falcão, F. D. S., & Canedo, E. D. (2026) · Empirical Software Engineering · doi:10.1007/s10664-026-10919-y
LLM-Based Automated Diagnosis Of Integration Test Failures At Google
Ziftci, C., Liu, R., Greene, S., & Dalloro, L. (2026) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3786583.3786863
A Road Map for Tamed Interactions with Large Language Models
Scotti, V., Keim, J., Hey, T., Metzger, A., Koziolek, A., & Mirandola, R. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3832776 · Cited by 2
“Alternatively, other models are distributed as coding assistants, like in the case of Copilot [47] or Code Llama [81], where fine-tuning is used to refine the model on source code generation, which is, however, a capability most pre-trained or assistant LLMs already have.”
Leveraging Low-Parameter LLMs for Self-Healing in Kubernetes-Based Container Orchestration
Wiesinger, Y., Engelhardt, M., & Laas, R. (2026) · International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS) · doi:10.1145/3788550.3794882
“• We evaluate the following LLMs: Mistral-7B-instruct [30], Llama-7B-chat [68], Codellama-7B-instruct [61] and Llama-3.”
InfluentialAn Empirical Study on Influence-Based Pretraining Data Selection for Code Large Language Models
Xing, C., Zeng, Z., Fang, G., Xie, R., Ye, W., & Zhang, S. (2026) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3794763.3794813
JavaBackports: A Dataset for Benchmarking Automated Backporting in Java
Kahapola, K., Galappaththi, S., Ranasinghe, D., Shariffdeen, R., de Silva, N., Perera, S., & Wickramanayake, S. (2026) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3793302.3793331
“Recent progress in Large Language Models (LLMs) for code comprehension and generation offers an opportunity to automate backporting [4, 7, 9, 23] as an Automated Program Repair (APR) task [10], but effective development of such tools requires high-quality, domain-specific datasets that reflect…”
BiVCoder: A Multi-Agent Framework for Code Generation via Bidirectional Code-Test Diagnosis
Li, X., Dong, J., Shi, W., Lü, W., & Du, X. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3818107
RL–LLMfuzzer: reinforcement learning–guided dual–model differential fuzzing for compiler testing
Li, D., Li, Y., Zhao, B., Xu, J., Yao, J., Wang, N., & Liu, L. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-01115-5
Efficiency Evaluation Framework for LLM-Generated Code
Avagian, D. A., & Ball, A. L. J. (2026) · Pattern Recognition and Image Analysis · doi:10.1134/s1054661826700306
“As a proof of concept for our efficiency evaluation benchmark, we generated solutions using six language models: Phi-1 [11], Phi-2 [17], Code Llama [31], QwQ-32B [30, 36], Qwen3 [29], and DeepSeek-V3 [7].”
Multi-agent collaborative optimization framework for integrated energy system scheduling
Qu, B., Jiang, H., Wang, S., Luo, Z., Lin, X., Tian, X., Li, J., Lin, X., & Zhong, W. (2026) · Journal of Zhejiang University. Science A · doi:10.1631/jzus.a2500629
STALL + : Boosting LLM-based Repository-level Code Completion with Static Analysis
Jun-wei, L., Chen, Y., Liu, M., Peng, X., & Lou, Y. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3846180 · Cited by 12
“Many LLMs ( e.g., CodeGen [56], StarCoder [41], and Code Llama [60]), have been widely applied in code-related tasks, such as code completion [23, 24, 82], test generation [20, 37], and program repair [12, 18, 53, 59].”
Training on Clean Data but Getting Backdoored Models! A Poisoning Attack on Code Encoders
Xiao, Y., Liu, X., Yang, Z., Bo, L., & Sun, X. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3773236
Filtering before Tuning: Robust Fine-Tuning of Large Code Models under Noisy Labels
Li, Z., Chen, Y., Yong, H., Lin, Y. K., Zhao, J., Xu, T., Pan, M., Zhang, T., & Li, X. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3787785
Bug Whisperer: Unleashing LLMs to Craft Context-Aware Tests Guided by Path Constraints and Bug Clues
Xia, X., Zhang, J., Hu, X., & Li, S. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3846194
Omri, M. N., & Louhichi, B. (2026) · Neural Computing and Applications · doi:10.1007/s00521-026-12365-9
From Greedy Steps to Global Optimization: Learning Sequential Test Suite Generation
Wang, G., Yang, C., Zhou, X., Sun, Z., Wang, B., Lo, D., & Hao, D. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832162 · Cited by 4
“1-8B [15], and CodeLlama-7B [44]) without any additional strategy and a fixed test budget 𝐾 = 5 utilized in previous work [20, 56].”
Cracking Query Bottlenecks: Towards Efficiency-Oriented Text-to-SQL Generation
Lin, L. G., Shen, Y., Bao, L., Wu, R., & Liu, Y. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832269
Liu, Y., 侯熙智, Yang, L., Liu, H., Zhu, C., Zhang, F., & Zuo, C. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832096
“The open-source models include: (1) the LLaMA series (LLaMA-13B [66], LLaMA-2-13B [67], and CodeLLaMA-13B [53]); (2) the Qwen series (Qwen-2.”
Re-evaluating Detection of Equivalent Mutants using LLMs: We Should Properly Measure How Far We Are
Tandon, A., Dündar, M. F., Gebru, M. G., Marinov, D., Lou, Y., & Wang, W. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832250
SAGE: Signal-Amplified Guided Embeddings for Vulnerability Detection
Shan, Z., Qian, X., Xin, J., Xu, M., Zhang, Y., Yang, Z., Wu, H., & Cheng, X. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832141 · Cited by 1
Automated Program Repair for UI-Centric Android Bugs: How Far Are We?
Mahmud, J., Pandey, S., Silva, N. D., Dipongkor, A. K., Wu, J., Chaparro, O., Fazzini, M., & Moran, K. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832272
“We used CodeLlama [59] instead of StarCoder [33], as prior work reports better repair performance [59].”
Yu, L., Wang, P., Xu, J. Q., Zhang, J., Wang, X., Ma, J., Yang, L., Deng, C., Wang, Z., & Zhang, F. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832094
“5-Coder-3B/7B/14B/32B-Instruct, Qwen3-Coder-30B-A3B), and the CodeLLaMA series [31] (CodeLlama-7B/13B/34B-Instruct).”
MAVCG: Model-Assisted Verified Code Generation from ODD Protocols via Dafny Specifications
Pánczél, P., & Gulyás, L. (2026) · Intelligent Robotics FAIR · doi:10.1145/3831600.3831609
What Breaks When LLMs Code? Characterizing Operational Safety Failures of Agentic Code Assistants
Hasan, A. A., & Biswas, S. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834393 · Cited by 5
“Large language models (LLMs) have rapidly evolved from static code completion tools into the decision-making core of autonomous coding agents [29, 44, 49, 74].”
Training datasets tailored for domain-specific large language models: challenges, methods and trends
Qiu, X., Ren, K., Cui, X., Liu, W., Gao, Z., Hu, J., & Liang, Q. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-01150-2
“It first applies an OpenAssistant reward model, then uses K-center-greedy selection to preserve diversity and finally estimates the necessity of seed samples through warm-up training on Llama 2-7B.”
InfluentialWu, Y., & Yang, Y. (2026) · doi:10.1145/3843777.3844528
“Modelstrainedonlargecodecorporacan producefunction-levelprograms,solvecompetitive-programming-styletasks,andsupportinteractiveprogrammingworkflows[2, 10,16].”
Evaluating Symbolic Feedback Granularity for LLM-Based Program Repair
Guo, T. (2026) · IEEE/ACM International Workshop on Automated Program Repair (APR) · doi:10.1145/3843776.3844675
“5-Coder have expanded the range of models available for code generation [8, 11, 16, 20].”
OrthCoder: Balancing Security and Utility in Secure Code Generation via Orthogonal Adaptation
Tian, M., Yuan, Y., Zhong, Z., & He, P. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3837499
“Iterative refinement approaches [2] repeatedly generate and revise code based on feedback signals (e.g., static analysis or execution results) to eliminate vulnerabilities.”
xuming, L., Zhang, C., Xu, J., Zhou, S., & Wang, J. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834342
“1-405B [19], Qwen3-Max [47], CodeLlama-7B [37], DeepSeek-Coder-236B [14], and StarCoder2-15B [26].”
CiL-Agent: Compiler-in-the-Loop LLM Agent for Memory-Safe C Program Repair
Shen, Y., Zhang, Y., Yao, J., Liu, M., & Dong, W. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3837493
Xu, J., Wu, Z., Peng, Y., Chong, C. Y., Meng, L., Liu, K., & Jia, X. (2026) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3832783.3834522
“…studies mainly explored general LLMs through zero-shot, few-shot, and prompt-enhanced settings [34, 36], while code-specialized foundation models such as Code Llama and WizardCoder [20, 25] are better described as important open-source backbones rather than direct vulnerability-detection studies.”
Metadata
- Authors
- Rozière, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., Rapin, J., Kozhevnikov, A., Evtimov, I., Bitton, J., Bhatt, M., Ferrer, C. C., Grattafiori, A., Xiong, W., Défossez, A., Copet, J., Azhar, F., Touvron, H., Martin, L., Usunier, N., Scialom, T., & Synnaeve, G.
- Affiliations
- École nationale des ponts et chaussées (FR)
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2308.12950
- OpenAlex
- W4386185625
- Cited by
- 531
- References
- 92
- Citations by year
- 2026: 65 · 2025: 232 · 2024: 216 · 2023: 18
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Keywords
- CodeLlama, code large language models, Llama 2, HumanEval, MBPP, code generation benchmarks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
