The DOI matches its DataCite record; 76 of the 92 references it cites were found in a registry or an index, and 1 are not checked yet.
Code Llama: Open Foundation Models for Code
Open accessAbstract
We release Code Llama, a family of large language models for code based on Llama 2 providing state-of-the-art performance among open models, infilling capabilities, support for large input contexts, and zero-shot instruction following ability for programming tasks. We provide multiple flavors to cover a wide range of applications: foundation models (Code Llama), Python specializations (Code Llama - Python), and instruction-following models (Code Llama - Instruct) with 7B, 13B, 34B and 70B parameters each. All models are trained on sequences of 16k tokens and show improvements on inputs with up to 100k tokens. 7B, 13B and 70B Code Llama and Code Llama - Instruct variants support infilling based on surrounding content. Code Llama reaches state-of-the-art performance among open models on several code benchmarks, with scores of up to 67% and 65% on HumanEval and MBPP, respectively. Notably, Code Llama - Python 7B outperforms Llama 2 70B on HumanEval and MBPP, and all our models outperform every other publicly available model on MultiPL-E. We release Code Llama under a permissive license that allows for both research and commercial use.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Given a new task, current LLMs consume and learn from the task data 'as-is' via finetuning or in-context learning [9, 10, 11, 12].”
From Self-Adapting Language Models (2025) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Code-based LLMs (e.g., CodeLlama(Roziere etal. 2023) and StarCoder(Li etal. 2023d; Lozhkov et al. 2024)) are particularly well suited to address this demand.”
From When LLMs meet cybersecurity: a systematic literature review (2025) · cites this paper as Roziere et al. 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“At the time of writing, only the CodeLlama models (Rozi` ere et al., 2023) are able to handle the very long contexts necessary.”
From SWE-bench: Can Language Models Resolve Real-World GitHub Issues? (2023) · cites this paper as Rozière et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Code Llama family are based upon Llama 2. It is pre-trained on a 500B token corpus with both code and natural language texts using the infilling objective.”
From Exploring the Capabilities of LLMs for Code-Change-Related Tasks (2025) · cites this paper as [67]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“which is a 7B decoder-only model trained on code synthesis.”
From Does Few-Shot Learning Help LLM Performance in Code Synthesis? (2024) · cites this paper as Roziere et al., 2023
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs have achieved remarkable proficiency at generating code in mainstream programming languages such as Python and JavaScript, yet this proficiency does not transfer to logic programming.”
From From Reasoning to Code: GRPO Optimization for Underrepresented Languages (2026) · cites this paper as Rozière et al. (2023)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“CodeLLaMA-34B [42] is an aligned version of LLaMA-234B, which is fine-tuned with 500B tokens of code data.”
From Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation (2024) · cites this paper as [42]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“At the same time, different LLMs possess unique strengths and specialize in various tasks aspects.”
From Mixture-of-Agents Enhances Large Language Model Capabilities (2024) · cites this paper as Roziere et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“There have been many previous works that have confirmed that LLMs do have good code comprehension capabilities (He et al. 2024; Luo et al. 2024; Roziere et al. 2023; Li et al. 2023d).”
From When LLMs meet cybersecurity: a systematic literature review (2025) · cites this paper as Roziere et al. 2023
The cited source says what the claim attributes to it.
In the citing paper
“Moreover, while numerous LMs have been implemented with a primary focus on code generation, most tend to underperform the best generalist LMs (Li et al., 2022; Le et al., 2022; Roziere et al., 2023; Austin et al., 2021; Xu et al., 2022).”
From Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation (2023) · cites this paper as Roziere et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Utilizing SWE-bench-train, we release two fine-tuned models, SWE-Llama 7 b and 13 b, based on the CodeLlama (Rozi` ere et al., 2023) model.”
From SWE-bench: Can Language Models Resolve Real-World GitHub Issues? (2023) · cites this paper as Rozière et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Prior studies showed that within the same LLM family, larger models are associated with better performance [59, 67, 73].”
From Exploring the Capabilities of LLMs for Code-Change-Related Tasks (2025) · cites this paper as [67]
The cited source says what the claim attributes to it.
In the citing paper
“The main experiments are performed on CODELLAMA (Roziere et al., 2023).”
From Does Few-Shot Learning Help LLM Performance in Code Synthesis? (2024) · cites this paper as Roziere et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Furthermore CODELLAMA is more specialized in coding tasks thus yielding slightly lower PERPLEX-ITY (TARGET) score compared to MISTRAL despite their comparable sizes.”
From Does Few-Shot Learning Help LLM Performance in Code Synthesis? (2024) · cites this paper as Roziere et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Among the LLMs tested, CODELLAMA (Roziere et al., 2023) benefits the most from good test case selection, hence we focus this work around CODELLAMA.”
From Does Few-Shot Learning Help LLM Performance in Code Synthesis? (2024) · cites this paper as Roziere et al., 2023
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Moreover, both of these tools aim to maximise the acceptance rate of the shown completions”
From A Transformer-Based Approach for Smart Invocation of Automatic Code Completion (2024) · cites this paper as [32]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Specifically, optimising only acceptance rate results in worse completions [32, 49].”
From A Transformer-Based Approach for Smart Invocation of Automatic Code Completion (2024) · cites this paper as [32]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Consistent with earlier research [32, 49], we acknowledge that these proxy metrics might not fully capture the usability of the interaction without potentially compromising it in some other way.”
From A Transformer-Based Approach for Smart Invocation of Automatic Code Completion (2024) · cites this paper as [32]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“and contains most of the significant features found in previous work [32].”
From A Transformer-Based Approach for Smart Invocation of Automatic Code Completion (2024) · cites this paper as [32]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×
“Effective handling of long sequences is a major topic of research in transformer-based language modeling (Vaswani et al., 2017).”
BackgroundBLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 3×
“( b) Difference in BLEU scores for single line code completion of long context models compared to their respective base models before fine-tuning.”
MethodInfluentialRoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · cited 1×
“To measure the toxicity of the generated outputs from each of the LLMs, we employ the default ToxiGen classifier, tuned on RoBERTa (Liu et al., 2019).”
MethodExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“…2023; Nguyen et al., 2023), many tasks related to code understanding or synthesis have been addressed since the early 2020s with NLP models adapted for code (Raffel et al., 2020; Feng et al., 2020; Guo et al., 2021; Wang et al., 2021; Ahmad et al., 2021), also see the survey by Xu & Zhu (2022).”
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087 · cited 2×
“The VADER produces sentiment scores between -1 and 1, where a positive (negative) score indicates a positive (negative) sentiment towards the population mentioned in the prompt.”
MethodLLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · cited 11×
“ChatGPT and GPT-4 (OpenAI, 2023), PaLM (Chowdhery et al., 2022) and Chinchilla (Hoffmann et al., 2022) are closed source, while BLOOM (Scao et al., 2022), OPT (Zhang et al., 2022b), and the seminal work of Llama are public (Touvron et al., 2023a).”
BackgroundMethodInfluentialSelf-citationKudo, T., & Richardson, J. (2018) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-2012 · Cited by 4,454 · cited 1×
“� 500B 20B 20B”
Transformer-XL: Attentive language models beyond a fixed-length context
Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1285 · Cited by 3,169 · cited 1×
“Long sequences in LLMs. Scaling Transformers and LLMs to long input sequences has attracted much recent interest (Dai et al., 2019; Beltagy et al., 2020; Yu et al., 2023; Ding et al., 2023).”
BackgroundTraining Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117 · cited 3×
“Collecting supervised data from human annotators or training from human feedback (Ouyang et al., 2022) is expensive for coding tasks as it requires input from professional developers.”
BackgroundMethodModel Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Research works printed but not linked: 6
- Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernandez Abrego, Junwhan Ahn, Jacob Austin, Paul Barham, Jan Botha, James Bradbury, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, Colin Cherry, Christopher A. Choquette-Choo, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, Mostafa Dehghani, Sunipa Dev, Jacob Devlin, Mark Díaz, Nan Du, Ethan Dyer, Vlad Feinberg, Fangxiaoyu Feng, Vlad Fienber, Markus Freitag, Xavier Garcia, Sebastian Gehrmann, Lucas Gonzalez, Guy Gur-Ari, Steven Hand, Hadi Hashemi, Le Hou, Joshua Howland, Andrea Hu, Jeffrey Hui, Jeremy Hurwitz, Michael Isard, Abe Ittycheriah, Matthew Jagielski, Wenhao Jia, Kathleen Kenealy, Maxim Krikun, Sneha Kudugunta, Chang Lan, Katherine Lee, Benjamin Lee, Eric Li, Music Li, Wei Li, YaGuang Li, Jian Li, Hyeontaek Lim, Hanzhao Lin, Zhongtao Liu, Frederick Liu, Marcello Maggioni, Aroma Mahendru, Joshua Maynez, Vedant Misra, Maysam Moussalem, Zachary Nado, John Nham, Eric Ni, Andrew Nystrom, Alicia Parrish, Marie Pellat, Martin Polacek, Alex Polozov, Reiner Pope, Siyuan Qiao, Emily Reif, Bryan Richter, Parker Riley, Alex Castro Ros, Aurko Roy, Brennan Saeta, Rajkumar Samuel, Renee Shelby, Ambrose Slone, Daniel Smilkov, David R. So, Daniel Sohn, Simon Tokumine, Dasha Valter, Vijay Vasudevan, Kiran Vodrahalli, Xuezhi Wang, Pidong Wang, Zirui Wang, Tao Wang, John Wieting, Yuhuai Wu, Kelvin Xu, Yunhan Xu, Linting Xue, Pengcheng Yin, Jiahui Yu, Qiao Zhang, Steven Zheng, Ce Zheng, Weikang Zhou, Denny Zhou, Slav Petrov, and Yonghui Wu. PaLM 2 Technical Report. arXiv:abs/2305.10403, 2023.
- Jaehun Jung, Peter West, Liwei Jiang, Faeze Brahman, Ximing Lu, Jillian Fisher, Taylor Sorensen, and Yejin Choi. Impossible distillation: From low-quality model to high-quality dataset & model for summarization and paraphrasing. arXiv:abs/2305.16635, 2023.
- Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019.
- Han Peng, Ge Li, Wenhan Wang, Yunfei Zhao, and Zhi Jin. Integrating tree path in transformer for code representation. In NeurIPS, pp. 9343-9354, 2021.
- Vighnesh Leonardo Shiv and Chris Quirk. Novel positional encodings to enable tree-based transformers. In NeurIPS, pp. 12058-12068, 2019.
- Michele Tufano, Dawn Drain, Alexey Svyatkovskiy, Shao Kun Deng, and Neel Sundaresan. Unit test case generation with transformers. arXiv:abs/2009.05617, 2020.
Cited by
Sorted by citations
Items 401–500 of 609
GPT Rotational Position Embedding for Length Extrapolation
Qu, Z. (2023) · International Conference on Machine Learning and Natural Language Processing (MLNLP) · doi:10.1145/3639479.3639514 · Cited by 2
“After this successively proposed NTK method fixed NTK [10], Dynamic NTK [11] adjustmentbase continue to train Base Code LLaMA [12], LLaMA2 Long [13] and other models give a better solution to the above problem.”
MethodDistilling Mathematical Reasoning Capabilities into Small Language ModelsPreprint
Zhu, X., Li, J., Liu, Y., Ma, C., & Wang, W. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4782551 · Cited by 2
Can large language models detect PII in code?Preprint
Gurjar, A., Camp, L. J., Ringenberg, T. R., Ma, X., & Chaora, A. C. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4619112 · Cited by 5
eidos: A modular approach to external function integration in LLMs
Aldana-Martín, J. F., Benítez‐Hidalgo, A., & Aldana-Montes, J. F. (2025) · SoftwareX · doi:10.1016/j.softx.2025.102290 · Cited by 2
Rethinking Verification for LLM Code Generation: From Generation to Testing
Ma, Z., Zhang, T., Cao, M., Liu, J., Zhang, W., Luo, M., Zhang, S., & Chen, K. (2025) · arXiv · doi:10.48550/arxiv.2507.06920 · Cited by 2
Self-citationiThelma: An Autonomous LLM Agent for Cyber Threat Hunting via Playbook-Driven Intelligence
Chen, N., Lin, R., Xie, D., Lin, H., & Chen, S. (2025) · 2025 IEEE Conference on Communications and Network Security (CNS) · doi:10.1109/cns66487.2025.11195050 · Cited by 2
“Models like Meta’s Code Llama [13], OpenAI’s GPT-4 [14], and Google’s Gemini [15] generate human-like text and executable code [16], enabling hunt script generation from natural language descriptions.”
Wang, Z., Chen, J., Wei, Z., Lin, L., & Tong, G. (2026) · Sensors · doi:10.3390/s26113502 · Cited by 2
“%) metric is adopted as the gold standard for code functional correctness, following the definition introduced together with HumanEval [15], which has since been used as the de facto functional metric in subsequent code LLM studies [16,17].”
Does Few-Shot Learning Help LLM Performance in Code Synthesis?
Xu, D., Xie, T., Xia, B., Li, H., Bai, Y., Sun, Y., & Wang, W. (2024) · arXiv · doi:10.48550/arxiv.2412.02906 · Cited by 3
“Current LLMs exhibit code generation capabilities through improved model design (Guo et al., 2024), training (Roziere et al., 2023), and chain-of-thought (Li et al., 2023).”
InfluentialMcEval: Massively Multilingual Code Evaluation
Chai, L., Liu, S., Yang, J., Yin, Y., Jin, K., Liu, J., Sun, T., Zhang, G., Ren, C., Guo, H., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.07436 · Cited by 73
“Large language models (LLMs) designed for code, such as Codex (Chen et al., 2021), CodeGen (Ni-jkamp et al., 2023), Code Llama (Rozière et al., 2023), DeepSeekCoder (Guo et al., 2024), and CodeQwen (Bai et al., 2023) excel at code understanding, completion, and generation tasks.”
MethodInfluentialSelf-citationLongCodeZip: Compress Long Context for Code Language Models
Shi, Y., Qian, Y., Zhang, H., Shen, B., & Gu, X. (2025) · arXiv · Cited by 7
“This need has motivated efforts to extend LLM context windows [14], [11], [15].”
FeedbackEval: A Benchmark for Evaluating Large Language Models in Feedback-Driven Code Repair Tasks
Dai, D., Liu, M., Li, A., Cao, J., Wang, Y., Wang, C., Peng, X., & Zheng, Z. (2025) · arXiv · doi:10.48550/arxiv.2504.06939 · Cited by 1
“Figure 5 compares Repair@1 across three datasets.”
From Reasoning to Code: GRPO Optimization for Underrepresented Languages
PENNINO, F., RAIMONDI, B., RONDELLI, M., GURIOLI, A., & GABBRIELLI, M. (2026) · Theory and Practice of Logic Programming · doi:10.1017/s1471068426100489 · Cited by 1
Leftover: Improving Large Language Model Inference Efficiency by Leveraging Idle Resources
Duan, X., & Ye, K. (2023) · International Conference on High Performance Big Data and Intelligent Systems (HPBD&IS) · doi:10.1109/hdis60872.2023.10499636 · Cited by 1
“With the advancement of algorithms, data, and computing power, large language models have demonstrated powerful performance across various application domains, including chatbots [1], [2], code generation [3], and document analysis [4], [5].”
(2024) · Journal of System and Management Sciences · doi:10.33168/jsms.2024.1015 · Cited by 3
FixCheck: A Tool for Improving Patch Correctness Analysis
Molina, F., Copia, J. M., & Gorla, A. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3685308 · Cited by 1
Improving Source Code Pre-Training via Type-Specific Masking
Zou, W., Li, Q., Li, C., Ge, J., Chen, X., Huang, L., & Luo, B. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3699599 · Cited by 2
“Code Llama [84], based on Llama 2 [94], performs well in infilling capabilities, support for large input contexts, and zero-shot instruction following ability for programming tasks.”
OWNER — Toward Unsupervised Open-World Named Entity Recognition
Genest, P.-Y., Portier, P.-É., Egyed‐Zsigmond, E., & Lovisetto, M. (2025) · IEEE Access · doi:10.1109/access.2025.3552122 · Cited by 2
“GoLLIE [20] uses Code-Llama [47] as its backbone and is fine-tuned on manually labeled datasets from the news and biomedical domains.”
SPRInT: Scaling Programmatic Reasoning for INstruction Tuning in mathematics
Yan, Y., Li, L., & Zhang, B.-W. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129868 · Cited by 1
Wang, K., Ding, Y., Jia, S., Ma, T., Zhang, Y., & Cao, B. (2024) · IEEE Smart World Congress (SWC) · doi:10.1109/swc62898.2024.00205 · Cited by 1
“Several prominent code generation models have emerged, such as Codex [7], StarCoder [8], and Code-Llama [9].”
CoServe: Efficient Collaboration-of-Experts (CoE) Model Inference with Limited Memory
Suo, J., Liao, X., Xiao, L., Ruan, L., Wang, J., Su, X., & Huo, Z. (2025) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3676641.3715986 · Cited by 1
“However, recent advances in machine learning research have shown that smaller experts 1 , such as Code Llama-Python 7B [32] and Flan-T5-XL 3B [4], can outperform general monolithic LLMs on specialized tasks, delivering higher performance and improved accuracy.”
Advanced code time complexity prediction approach using contrastive learning
Park, S., Hahn, J., Orwig, E., Ko, S., & Han, Y.-S. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110631 · Cited by 1
On the Applicability of Code Language Models to Scientific Computing Programs
Zhao, Q., Liu, F., Long, X., Wu, C., & Zhang, L. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3564599 · Cited by 2
“Recently, Meta released CodeLlama [48], built upon Llama 2, providing support for large input contexts and zero-shot instruction following ability for various programming tasks, excelling among open-sourced models.”
Enhancing Code Generation Through Retrieval of Cross-Lingual Semantic Graphs
Jiang, Z., Shi, Z., Gao, X., & Xiong, Y. (2024) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec65559.2024.00026 · Cited by 1
“Despite substantial advancements by code language models [1]–[6], which have been pre-trained on extensive code corpora, they continue to face challenges such as knowledge updating and avoiding erroneous outputs, commonly referred to as ”hallucinations” [7]–[10].”
Exploring continual learning in code intelligence with domain-wise distilled prompts
Liu, S., Keung, J., Yang, Z., Liu, F., Zhang, F., & Sun, Y. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107775 · Cited by 1
Accessible Smart Contracts Verification: Synthesizing Formal Models with Tamed LLMs
Corazza, J., Gavran, I., Moreira, G., & Neider, D. (2025) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst62969.2025.10989026 · Cited by 2
“LLMs optimized for code generation showed stunning re-sults [17], [18].”
UniGenCoder: Merging SEQ2SEQ and SEQ2TREE Paradigms for Unified Code Generation
Shao, L., Yan, Y., Poshyvanyk, D., & Su, J. (2025) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier66352.2025.00020 · Cited by 3
“Therefore, in future work, we plan to extend our work to large-scale LLMs ( e.g., , Code Llama [21], StarCoder [22]) to achieve superior performance as long as they are open-source.”
Code Ranking with Structure Awareness Contrastive Learning
Huang, H., Cao, L., Wang, J., Yu, T., & Cai, Y. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00053 · Cited by 1
“However, the rapid growth in the size of models, such as Code Llama [18], has made fine-tuning increasingly resource-intensive and less practical.”
InfluentialCLCoSum: Curriculum Learning-Based Code Summarization for Code Language Models
He, H., Wang, J., Cao, L., & Cai, Y. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00035 · Cited by 1
“In recent years, with the success of pre-trained models in NLP, similar work has emerged in the field of code intelligence [15]–[19], [28], [67].”
Viera, E., Candia, D. A., Soto, I., Lagos, C., Játiva, P. P., Carrasco, R., Azurdia-Meza, C. A., & Sánchez, I. (2025) · IEEE Access · doi:10.1109/access.2025.3581790 · Cited by 1
DeCoMa: Detecting and Purifying Code Dataset Watermarks through Dual Channel Code Abstraction
Xiao, Y., Chen, Y., Ma, S., Huang, H., Fang, C., Chen, Y., Sun, W., Zhu, Y., Zhang, X., & Chen, Z. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728952 · Cited by 1
“Specifically, AC uses a well-trained backdoored model to compute the representation values of inputs for each label.”
InfluentialCodeFlow: Program Behavior Prediction with Dynamic Dependencies Learning
Le, C. A., Phan, H. N., Phan, H. N., Nguyen, T. N., & Bui, N. D. Q. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00029 · Cited by 6
Extracting Fix Ingredients using Language Models
Prenner, J. A., & Robbes, R. (2025) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1109/forge66646.2025.00028 · Cited by 1
“Code Llama [42] was trained with a context size 16k (and can theoretically handle up to 100k input tokens), while Mistral’s Mixtral [47] has a context size of 32k. Anthropic’s Claude family of models (in particular Claude 2 and Claude 3) support context windows of up to 200k tokens [48].”
Contextual Augmented Multi-Model Programming (CAMP): A Local-Cloud Copilot Solution
Wang, Y., Guo, S., & Tan, C. W. (2025) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai64502.2025.00122 · Cited by 1
“Cloud-based tools leveraging LLMs, such as Codeium [10], GitHub Copilot [11], [12], OpenAI ChatGPT [13], Amazon CodeWhisperer [14], and Meta’s Code Llama [15], provide users with access to powerful LLM services and online resources through dedicated APIs on demand.”
An end-to-end approach for fixing concurrency bugs via SHB-based context extractor
Li, Z., Yi, Q., Xiao, K., Ji, Z., & Liang, H. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.112888 · Cited by 1
“To handle longer contexts, models such as DeepSeek-V3 [96], the Llama3 series [95], and QWen2.”
Chen, L., Ahmed, N. K., Capotă, M., Willke, T., Hasabnis, N., & Jannesari, A. (2025) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps64566.2025.00055 · Cited by 2
“Code LLMs such as StarCoder [3], CodeLLama [4], and DeepSeek-coder [5] have demonstrated remarkable capabilities across a wide range of programming tasks, from code generation and completion to bug detection and API recommendation.”
Advanced code slicing with pre-trained model fine-tuned for open-source component malware detection
Wang, Y., Pang, S., Fan, Z., Shang, S., Yao, Y., Jiang, Z., & Liu, B. (2025) · The Computer Journal · doi:10.1093/comjnl/bxaf029 · Cited by 1
Generative AI tools for accelerated software engineering
Kanmani, P., Prabha, K. C., Veerandeswari, J., & Manjula, M. (2025) · Advances in computers · doi:10.1016/bs.adcom.2025.06.008 · Cited by 3
Sundaram, G., Venktesh, B., V, & Sundharakumar, K. B. (2025) · International Conference on Emerging Technologies in Computing and Communication (ETCC) · doi:10.1109/etcc65847.2025.11108633 · Cited by 1
“Subsequent papers such as Code Llama [7] test DocString generation on CodeXGLUE [8] using Bilingual Language Understanding (BLUE) score, while the other three models used in the study Code Gemma [9], DeepSeek Coder [10], and Qwen 2.5 Coder [11] does not state information related to DocString…”
InfluentialWatermarking LLMs — Challenges and Opportunities in Electronic Design Automation
DeLorenzo, M. C., Tieu, P., Chen, C., Gohil, V., & Rajendran, J. (2025) · IEEE International Conference on Omni-layer Intelligent Systems (COINS) · doi:10.1109/coins65080.2025.11125763 · Cited by 1
“Current state-of-the-art models, including GPT-4o [1], Meta’s Llama [4], and Anthropic’s Claude [5], have demonstrated potential in coding performance.”
Evaluation and Improvement of Test Selection for Large Language Models
Quan, L., Jin, W., Hu, Q., Cordy, M., Huang, Y., Ma, L., & Li, X. (2025) · Journal of Software Evolution and Process · doi:10.1002/smr.70057 · Cited by 4
Mao, K., Hu, B., Lin, R., Li, Z., Lu, G., & Zhang, Z. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1660912 · Cited by 4
“Representative models such as Codex, CodeLlama, DeepSeek Coder, and StarCoder (Guo et al., 2024; Roziere et al., 2023; Li et al., 2023) are trained and aligned on large-scale code corpora, enabling them to abstract program structures from complex semantic descriptions.”
Ma, H., Ji, Z., Al-Hashimy, T., Allen, A., Cen, N., Chen, O., Chen, Y., Chen, Y., Cheng, T., Gu, Y., et al. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2521062122 · Cited by 1
Output format biases in the evaluation of large language models for code translation
Macedo, M., Tian, Y., Cogo, F. R., & Adams, B. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10768-1 · Cited by 2
“The detailed design concerns for this prompt can be found in Section 5.”
On the Evaluation of Large Language Models in Unit Test Evolution (Experience Paper)
None, Zhang, J., Niu, Y., & Zhou, B. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832282 · Cited by 1
Li, D., Li, Y., Chen, X., & Zhang, W. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3607 · Cited by 1
Auditing unauthorized training data from AI generated content using information isotopes
Qi, T., Yin, J., Cai, D., Xie, Y., Wang, H., Hu, Z., Yang, P., Nan, G., Zhou, Z., Wu, C., et al. (2026) · Nature Communications · doi:10.1038/s41467-026-68862-x · Cited by 3
“Speci fi cally, we utilize the Stack dataset 34 , which comprises open-source code repositories collected from GitHub and is widely adopted in training code-focused AI models (e.g., Code LLaMA 35 ).”
Ahmed, S., & Sadeque, F. (2026) · JMIR Medical Informatics · doi:10.2196/82545 · Cited by 1
Practical LLM-Based Function-Level Automated Program Repair: How Far Are We?
Xiang, J., Xu, X., Kong, F., Wu, M., Zhan, Z., Zhang, H., & Zhang, Y. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3812804 · Cited by 1
“1 [21] since they have conducted the instruction fine-tuning [97] and can better follow the APR prompt instruction.”
Nabil, H. A. M., Lutfi, S. L., & Mulki, H. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3878 · Cited by 1
Code–Test Co-translation: Towards Practical and Effective Program Migration in the Wild
Li, X., Xie, X., Wu, J., Liu, T., & Wang, H. (2026) · Proceedings of the ACM on Programming Languages · doi:10.1145/3839490
“In the current era of LLMs, models have been widely applied to code translation tasks [6, 15, 25], showing notably high success rates.”
Wang, R., Todd, G., Yuan, X., Xiao, Z., Côté, M.-A., & Jansen, P. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.830 · Cited by 17
Sensi-Bert: Towards Sensitivity Driven Fine-Tuning for Parameter-Efficient Language Model
Kundu, S., Sridhar, S. N., Szankin, M., & Sundaresan, S. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447812 · Cited by 2
“Recently, the advent of generative pre-trained (GPT) [6] and various other large language models [7] has further pushed the boundary of applications with the pre-trained language models, ranging from AI for science to code generation [8].”
BackgroundTowards Safe, Secure, and Usable LLMs4Code
Al-Kaswan, A. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3639478.3639803 · Cited by 2
“Many state-of-the-art open source LLMs4Code such as CodeLlama [9] and WizardCoder [10] cannot be executed on consumer GPUs with less than 32GB of VRAM 1 .”
BackgroundExamples out of Thin Air: AI-Generated Dynamic Context to Assist Program Comprehension by Example
Mattis, T., Krebs, E., Rinard, M., & Hirschfeld, R. (2024) · International Conference on Art, Science, and Engineering of Programming (<Programming>) · doi:10.1145/3660829.3660845 · Cited by 1
Enhancing Parallelization with OpenMP through Multi-Modal Transformer Learning
Chen, Y., Yuan, H., Hou, F., & Hu, P. (2024) · International Conference on Computer Engineering and Application (ICCEA) · doi:10.1109/iccea62105.2024.10603704 · Cited by 1
“In the field of code processing, many code models based on Transformer have been proposed, e.g., Code Llama[3], StarCoder[4] .”
MethodTowards LLM4PCG: A Preliminary Evaluation of Open-Weight Large Language Models Beyond ChatGPT4PCG
Taveekitworachai, P., Xia, Y., Suntichaikul, P., & Thawonmas, R. (2024) · IEEE Conference on Games (CoG) · doi:10.1109/cog60054.2024.10645646 · Cited by 1
Trofimova, E., Sataev, E., & Ustyuzhanin, A. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2328
“The selection of GPT-3.”
Fine-Tuning CodeLlama to Fix Bugs
Szalontai, B., Vadász, A., Márton, T., Pintér, B., & Gregorics, T. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-3442-9_34 · Cited by 2
VulnMiner: A comprehensive framework for vulnerability collection from C/C++ source code projects
Bhandari, G. P., Gavrić, N., & Shalaginov, A. (2024) · Software Impacts · doi:10.1016/j.simpa.2024.100713 · Cited by 1
KOSMOS-E : Learning to Follow Instruction for Robotic Grasping
Wang, Z., Wu, X., Huang, S., Li, D., Wang, W., Ma, S., & Wei, F. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10802219 · Cited by 1
“…pretrained models, e.g., Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) has witnessed remarkable progress on various tasks [1], [2], [3], [4], e.g., reasoning [5], [6], planning [7], [8], code generation [9], [10] and open-vocabulary visual interactions [11], [12], [13].”
Optimal RoPE extension via Bayesian Optimization for training-free length generalization
Zhang, X., Hu, S., Zhao, W., Wang, H., Han, X., He, C., Zeng, G., Liu, Z., & Sun, M. (2025) · AI Open · doi:10.1016/j.aiopen.2025.01.002 · Cited by 1
Sterbini, A., & Temperini, M. (2024) · International Conference on Information Technology Based Higher Education and Training (ITHET) · doi:10.1109/ithet61869.2024.10837616 · Cited by 1
CIPAC: A framework of automated software construction based on collective intelligence
Liu, J., Zhang, Y., Li, Y., Ma, T., & Dong, W. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112335 · Cited by 2
Self-citationShawon, A., Liscano, R., Azim, A., Sundaresan, V., & Chang, Y.-K. (2024) · IEEE International Conference on Collaborative Advances in Software and Computing (CASCON) · doi:10.1109/cascon62161.2024.10837708 · Cited by 1
Almaghthawi, A., & Yafooz, W. M. S. (2025) · International Conference on Advanced Innovations in Smart Cities (ICAISC) · doi:10.1109/icaisc64594.2025.10959494 · Cited by 1
“The models have been trained extensively and demonstrate competitive performance [38].”
Towards Trustworthy LLMs for Code: A Data-Centric Synergistic Auditing Framework
Wang, C., Chen, Z., Li, T., Zhang, Y., & Liu, Y. (2025) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier66352.2025.00017 · Cited by 8
Memorization in LLM-Based Program Repair
Kong, J., Cheng, M., & Xie, X. (2025) · IEEE/ACM International Workshop on Automated Program Repair (APR) · doi:10.1109/apr66717.2025.00011 · Cited by 1
“5 [15] and CodeLlama-7b-Instruct [16].”
CODEMORPH: Mitigating Data Leakage in Large Language Model Assessment
Rao, H., Zhao, Y., Zhu, W., Xiao, L., Wang, M., & Wang, H. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion66252.2025.00081 · Cited by 1
“As Large Language Models for Code (Code LLMs) continue to advance [27], [17], [29], researchers test these LLMs by having them complete a series of code tasks on evaluation datasets, scoring their performance based on task completion and evaluation metrics to represent the LLM’s capabilities.”
Cmdesum: A Cross-Modal Deliberation Network for Code Summarization
Liao, Z., Liu, X., Lan, P., Yu, S., & Liu, P. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00034 · Cited by 1
“• Code Llama [44] is a large language model developed by Meta for programming languages.”
Herbold, S., Knieke, C., Rausch, A., & Schindler, C. (2025) · doi:10.1109/nse66660.2025.00008 · Cited by 3
“On the one side, we have Large Language Models (LLMs), e.g., GPT-4 [4], LLAMA [5], CodeLLAMA [6], among many others.”
Automated Discovery of Integral with Deep LearningPreprint
Yin, X. (2024) · Research Square · doi:10.21203/rs.3.rs-3942342/v1 · Cited by 1
Inference Acceleration for Large Language Models on CPUsPreprint
Ditto, P. S., & Jithin, V. G. (2024) · Preprints.org · doi:10.20944/preprints202402.1702.v1 · Cited by 1
Predicting typeScript type annotations and definitions with machine learning
Yee, M. (2024) · doi:10.17760/d20653005 · Cited by 1
Not Explainable but Verifiable
Burgess, P., & Shareghi, E. (2025) · Oxford University Press eBooks · doi:10.1093/9780198945215.003.0093 · Cited by 1
An, Y., Lu, C., Zhu, K., Yu, T., Zhao, C., Wu, H., Tang, M., & Wang, J. (2026) · arXiv · doi:10.48550/arxiv.2605.08840
“These models have enabled breakthroughs in various tasks, such as document summarization(Zhang et al., 2024a), multi-turn dialogues (Du et al., 2021), retrieval augmentation (Yao et al., 2022), and code generation (Roziere et al., 2023).”
Multi-LCB: Extending LiveCodeBench to Multiple Programming Languages
Ivanova, M., Zadorozhny, P., Levichev, R., Petrov, I., Pavel, A., Lopatin, I., Kutalev, A., & Babaev, D. (2026) · arXiv · doi:10.48550/arxiv.2606.20517 · Cited by 1
“Large language models (LLMs) have recently demonstrated impressive capabilities in code-related tasks (Ridnik et al., 2024; Lozhkov et al., 2024; Roziere et al., 2023; Li et al., 2022; Nijkamp et al., 2022), powering applications such as AI-assisted programming, automated debugging, and code…”
Jiang, J., Shen, J., Kim, S., Yoo, K. M., Kim, J., & Kim, S. (2026) · arXiv · doi:10.48550/arxiv.2603.05863 · Cited by 8
“5-Coder-7B-Instruct (Hui et al., 2024), Seed-Coder-8B-Instruct (Seed et al., 2025), DeepSeek-Coder-7B-Instruct (Guo et al., 2024), CodeGemma-7B-IT (Team et al., 2024), and CodeLlama-7B-Instruct (Roziere et al., 2023), LeDex-RL-7B (Jiang et al., 2024), and DeepCoder-14B-Preview (Luo et al., 2025).”
Demonstration Attack against In-Context Learning for Code Intelligence
Ge, Y., Sun, W., Lou, Y., Fang, C., Zhang, Y., Li, Y., Zhang, X., Liu, Y., Zhao, Z., & Chen, Z. (2024) · arXiv · doi:10.48550/arxiv.2410.02841
“LLMs trained on large-scale code repositories, such as Star-Chat [42] and CodeLLama [34], have demonstrated exceptional performance in code intelligence tasks like code generation [8, 45], code summarization [1, 10, 38, 39] and code translation [22, 44].”
BackgroundMethodInfluentialRethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?
Li, W., Lin, Y., Xia, M., & Jin, C. (2025) · arXiv · doi:10.48550/arxiv.2502.00674 · Cited by 62
“…with automatic verifiers available, such as math (Hendrycks et al., 2021) and code (Chen et al., 2021), simply sampling LLMs multiple times can significantly improve the pass@k metric and hence boost the success rate of solving the tasks (Roziere et al., 2023; Li et al., 2022; Brown et al., 2024).”
Zhou, C., Zhang, X., Song, D., Chen, X., Gu, W., Ma, H., Tian, Y., Zhang, M., & Hu, L. (2025) · arXiv · doi:10.48550/arxiv.2502.09183 · Cited by 18
On the Difficulty of Selecting Few-Shot Examples for Effective LLM-based Vulnerability Detection
Hannan, M. A., Ni, R., Zhang, C., Jia, L., Mangal, R., & Pasareanu, C. S. (2026) · Proceedings 2026 Workshop on LLM Assisted Security and Trust Exploration · doi:10.14722/last-x.2026.23025
“Early research in this category has often focused on models like CodeBERT [35] and GraphCode-BERT [36], with more recent work exploring larger foundation models such as Qwen [37]–[40] and LLaMA [41]–[43].”
Post-Incorporating Code Structural Knowledge Into Pretrained Models via ICL for Code Translation
Du, Y., Sun, H., & Li, M. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3605768
“Recently, large language models (LLMs) have demonstrated exceptional performance across various software mining tasks [1], [2], [3], [4], [5], [6], [7].”
Bhattarai, M., Santos, J. E., Jones, S., Biswas, A., Alexandrov, B., & O'Malley, D. (2024) · arXiv
“We conducted extensive experiments using various open language models, including Starcoder [7], Llama3-70B In-struct [8], CodeLlama-34B Instruct [9], Granite-34B Code Instruct [10], Mistral 8x22B [11] and Codestral [12] and commercial models such as GPT-3.”
Are Performance-Optimization Benchmarks Reliably Measuring Coding Agents?
Chen, Z., Sun, Z., Shi, Y., Lo, D., & Jiang, L. (2026) · arXiv · doi:10.48550/arxiv.2607.01211 · Cited by 9
“In the remaining text of this paper, we use submission for the evaluated agent configuration and use model names only as shorthand labels for the underlying model component [20]–[22].”
SE-Blackboard: A Shared-State Architecture for Multi-Agent Software Engineering Pipelines
Liu, E., Zhu, Q., & Dong, X. (2026) · IEEE Access · doi:10.1109/access.2026.3695435
“Recent studies have introduced a range of multi-agent software engineering (SE) systems that differ in architectural design and interaction style, which typically rely on the code-generation strengths of foundation models such as Codex [6] and Code Llama [12].”
Exploring the Capabilities of LLMs for Code Change Related Tasks
Fan, L., Liu, J., Liu, Z., Lo, D., Xia, X., & Li, S. (2024) · arXiv · Cited by 41
“1 For example, Meta released the Code Llama models [67], which are initialized from the Llama 2 models [73] and trained on 500B tokens from a code-heavy dataset.”
InfluentialSelf-citationAlgoVeri: An Aligned Benchmark for Verified Code Generation on Classical Algorithms
Zhao, H., Yang, Z., Li, J., He, D., Li, Z., Jin, C., Veeravalli, V. V., Gupta, A., & Arora, S. (2026) · arXiv · doi:10.48550/arxiv.2602.09464
“Large language models (LLMs) are widely used to generate code from natural language instructions (Chen et al., 2021; Roziere et al., 2023).”
TwinMem-Agent: Episodic and Semantic Memory for Repository-Level Program Repair
Wu, B., Luo, Z., Mu, F., & Xiang, L. (2026) · Proceedings of the 34th ACM International Conference on the Foundations of Software Engineering · doi:10.1145/3803437.3807664
BIRD: Business Insights and Recommendations Developer using Large Language ModelsPreprint
Karunanithi, S. (2024) · doi:10.36227/techrxiv.170594587.74941582/v1
Natural Language Conditioned Planning of Complex Robotics Tasks
Zinars, T. E., Vismanis, O., Racinskis, P., Ārents, J., & Greitāns, M. (2024) · River Publishers eBooks · doi:10.1201/9781003478713-6
Enhancing Biosecurity with Language Models: Defining Research DirectionsPreprint
Chen, M., Holub, M., & Tice, C. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4772574
Predicting Return Types from Method NamesPreprint
Werger, O., Kahle, J., & Hesenius, M. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4807693
A Survey on Code Pre-Trained Models for Neural Program RepairPreprint
Zhan, S., Wang, X., Wei, D., & Cao, X. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4831589
Chatsos: Vector Database Augmented Generative Question Answering Assistant in Safety EngineeringPreprint
Tang, H., Chen, D., Chu, Q., & Liu, Z. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4863282
Translating and Optimising Computational Microscopy Algorithms with Large Language Models
Guzzi, F., Kourousias, G., Pugliese, R., Gianoncelli, A., & Billè, F. (2024) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro60963.2024.10569791
“…interest in 2023: in [20] the authors propose i) a dataset, ii) a framework to estimate the code speedup and iii) a model based on CodeLlama13B [21], that is a specific foundational model trained for code-based prompts; similarly, in [12] the authors analyze the limits of current code…”
BackgroundA Comprehensive Evaluation of Large Language Models for Method Name PredictionPreprint
Qian, H., Xu, T., Ding, Z., Liu, W., Zhu, S., Yin, J., Feng, X., Fang, C., Zhao, J., & Sun, W. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4886577
Metadata
- Authors
- Rozière, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., Rapin, J., Kozhevnikov, A., Evtimov, I., Bitton, J., Bhatt, M., Ferrer, C. C., Grattafiori, A., Xiong, W., Défossez, A., Copet, J., Azhar, F., Touvron, H., Martin, L., Usunier, N., Scialom, T., & Synnaeve, G.
- Affiliations
- École nationale des ponts et chaussées (FR)
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2308.12950
- OpenAlex
- W4386185625
- Cited by
- 531
- References
- 92
- Citations by year
- 2026: 65 · 2025: 232 · 2024: 216 · 2023: 18
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Keywords
- CodeLlama, code large language models, Llama 2, HumanEval, MBPP, code generation benchmarks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
