Post-Incorporating Code Structural Knowledge Into Pretrained Models via ICL for Code Translation
Open accessThe DOI matches its Crossref record; 68 of the 112 references it cites were found in a registry or an index, and 26 are not checked yet; 9 of the 41 sources its claims cite were checked for support.
Abstract
Code translation migrates codebases across programming languages. Recently, large language models (LLMs) have achieved significant advancements in software mining. However, handling the syntactic structure of source code remains a challenge. Classic syntax-aware methods depend on intricate model architectures and loss functions, rendering their integration into LLM training resource-intensive. This paper employs in-context learning (ICL), which directly integrates task exemplars into the input context, to post-incorporate code structural knowledge into pre-trained LLMs. We revisit exemplar selection in ICL from an information-theoretic perspective, proposing that list-wise selection based on information coverage is more precise and general objective than traditional methods based on combine similarity and diversity. To address the challenges of quantifying information coverage, we introduce a surrogate measure, Coverage of Abstract Syntax Tree (CAST), measuring maximum subtree coverage between ASTs of test source code and exemplars. Furthermore, we formulate the NP-hard CAST maximization for exemplar selection and prove that it is a standard submodular maximization problem. Therefore, we propose a greedy algorithm for CAST submodular maximization, which theoretically guarantees a (1 − 1/e)-approximate solution in polynomial time complexity. Our method is the first training-free and model-agnostic approach to post-incorporate code structural knowledge into existing LLMs at test time. Experimental results show that our method significantly improves LLMs performance in code translation and reveals two meaningful insights: 1) Code structural knowledge can be effectively post-incorporated into pre-trained LLMs during inference, despite being overlooked during training; 2) Scaling up model size or training data does not lead to the emergence of code structural knowledge, underscoring the necessity of explicitly considering code syntactic structure.
Verified 2026-11-22 · 26 references not checked yet
Overview
CoChat Summary
## TL;DR
The paper proposes CAST, a training-free, model-agnostic in-context learning retrieval method that post-incorporates code structural knowledge into pretrained LLMs for code translation. Across code translation benchmarks, CAST-F generally outperforms similarity- and diversity-based exemplar selection, and the paper reports that scaling model size alone does not make structural knowledge emerge sufficiently.
## Key findings
- On 5-shot code translation with Qwen2-7B, CAST-F achieved an average CA of 64.28%, higher than Random (61.28%), LD (62.33%), AST-ED (63.12%), BM25 (62.09%), CodeBERT (61.91%), and Diversity (62.68%).
- For Gemma2-27B, CAST-F reached 70.03% average CA, exceeding Random (68.81%) and Diversity (68.13%); for Qwen2.5-72B, CAST-F reached 71.33% average CA versus Diversity at 70.19%.
- On AVATAR with Qwen2-7B, CAST-F achieved the best reported computational accuracy in the table: 47.79% for Java→Python and 50.40% for Python→Java, and the paper says it also gave the best overall CodeBLEU scores.
- For code summarization on CodeXGLUE, CAST-F achieved the highest BLEU and Rouge-L scores across nearly all languages and metrics; for example, on Java it scored 2.42 BLEU / 9.90 Rouge-L, and on Ruby 6.71 BLEU / 14.47 Rouge-L.
- In the failure analysis for Python→Java 20-shot translation, CAST-F had the lowest Wrong Answer rate (26.41%) and the highest Accept rate (58.10%).
## Methods
- The paper formulates code translation as ICL with exemplar selection and proposes CAST (Coverage of AST), a surrogate score for information coverage based on the ratio of maximum covered subtrees between the test code AST and exemplar ASTs.
- It uses tree-sitter to extract ASTs, keeps only node types, computes subtree fingerprints by hashing, builds a co-occurrence matrix between test subtrees and database samples, and selects exemplars by maximizing CAST.
- CAST maximization is shown to be NP-hard but a non-negative, monotone, submodular maximization problem; the paper uses a greedy algorithm with a theoretical (1 - 1/e) approximation guarantee and evaluates it on 948 parallel functions, focusing on the 568 functions with unit tests, plus AVATAR and CodeXGLUE code summarization benchmarks.
## Limitations
- The paper explicitly states that improving CAST may require considering more advanced code structure beyond AST, such as CFG and DFG, and suggests a feedback loop during inference as future work.
- The threat analysis states a possible internal validity threat from LLM data leakage due to training/testing overlap, though it says prior work suggests the threat is limited.
- It also notes an external validity threat from dataset quality and an external validity threat from implementation differences across compared techniques; these are mitigated by using state-of-the-art models and a cleaned, widely used dataset, but the threat itself is still stated.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures












Verification
Claims the paper makes
8 Valid · 8 Not valid · 19 Not checked · 126 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“In-context learning (ICL) provides an effective and efficient solution of integrating new domain-specific knowledge into off-the-shelf LLMs without additional training by directly incorporating task exemplars into the input context [16].”
What the sources say
- Supportedcites [16] [16] M. Luo, X. Xu, Y. Liu, P. Pasupat, and M. Kazemi, 'In-context learning with retrieved demonstrations for language models: A survey,' Transactions on Machine Learning Research , 2024. Survey Certification. - source
The Introduction (Sec. 1) and Sec. 2.1 state that off-the-shelf pre-trained LLMs adapt to a new task/domain by only seeing a few task examples in the input, 'as opposed to needing additional training data or fine-tuning', and the 'Benefits of ICL' paragraph adds that ICL requires fewer resources and less data, is computationally cheaper than PEFT, and leaves parameters unchanged — covering the 'effective and efficient … without additional training' framing. The 'domain-specific knowledge' wording is a light rephrasing of the source's new-task/downstream-domain adaptation (also reflected in Sec. 6 on cross-domain/cross-lingual demonstration use), so the attribution holds at the level of content. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Although generally NP-hard, a greedy algorithm that iteratively maximizes the marginal gain offers a theoretical (1 -1 /e ) -approximate solution in polynomial time.”
What the sources say
- Not checked — no full text of the sourcecites [29] [29] G. L. Nemhauser and L. A. Wolsey, 'Best algorithms for approximating the maximum of a submodular set function,' Mathematics of operations research , vol. 3, no. 3, pp. 177-188, 1978. - source
- Supportedcites [30] [30] H. Sun, S. Lu, H. Wang, Q.-G. Chen, Z. Xu, W. Luo, K. Zhang, and M. Li, 'MDP3: A training-free approach for list-wise frame selection in video-LLMs,' arXiv preprint arXiv:2501.02885 , 2025. - source
Appendix A.2 (p. 14) states submodular maximization is NP-hard in general and that maximizing a monotone submodular function under a cardinality constraint admits a (1-1/e)-approximation greedy algorithm, with Theorem 1 defining the greedy procedure as iteratively selecting the item that maximizes the marginal gain from S = ∅ until size k. Section 3.2 (p. 4) repeats this at the level of the citing sentence: the problem is 'generally NP-hard' and the greedy algorithm 'offers polynomial complexity and guarantees a (1-1/e)-approximation'. Non-negativity is carried by Definition 1 (p. 13), which takes f : 2^Ω → R≥0, so all qualifiers in the paraphrase (non-negative, monotone, submodular, polynomial time) are present in our copy. (LLM, conf=0.92)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“explicitly incorporating structural knowledge into models enhances their perceptual understanding of code, thereby aiding software mining tasks [14], [15], including code translation [9], [13].”
What the sources say
- Supportedcites [14] [14] S. Liu, C. Gao, S. Chen, L. Y. Nie, and Y. Liu, 'ATOM: Commit message generation based on abstract syntax tree and hybrid ranking,' IEEE Transactions on Software Engineering , vol. 48, no. 5, pp. 18001817, 2022. - source
The Motivation (2.2, p. 3) states that program comprehension tasks — vulnerability identification, function name prediction, source code summarization — 'have utilized code structure such as Abstract Syntax Tree (AST) to learn code semantics and good performance has been demonstrated', and 2.3 (p. 3) adds that 'Adopting AST in code comprehension has been proved to get the state-of-the-art performance'. The Abstract (p. 1) and 5.1 (p. 11) show the paper itself explicitly incorporating AST because it 'can be important for capturing code semantics', yielding a 30.72% BLEU-4 gain on commit message generation, which the paper frames as NMT translation of code changes into language (6.1, p. 15; 6.2, p. 15 also notes Wan et al. incorporating an AST to translate code snippets). This broad background paraphrase is thus treated by our copy, at the level of generality the citing sentence uses. (LLM, conf=0.80)
- Supportedcites [15] [15] L. Gong, M. Elhoushi, and A. Cheung, 'AST-T5: Structure-aware pretraining for code generation and understanding,' in Proceedings of the 41st International Conference on Machine Learning , pp. 15839-15853, 2024. - source
AST-T5's Abstract and 5.2 Main Results state that structure-awareness aids both code understanding and code-to-code translation: 'The benefits of being structure-aware are further exemplified by AST-T5's leading results in Clone Detection, where it surpasses CodeT5 by 3 points' and 'Structure-awareness makes AST-T5 particularly powerful in code-to-code tasks, surpassing CodeT5 by ... 3 points in exact match score for Java-C# Transpilation in CodeXGLUE.' The Introduction adds the general point that 'leveraging the Abstract Syntax Tree (AST) of code can significantly improve performance on code-related tasks,' and the ablation table (Section 5) shows AST-aware segmentation/corruption raising the Understanding columns. The paper does not use the phrase 'software mining,' but its evaluated tasks (clone detection on BigCloneBench, defect detection, transpilation) are exactly those, so the attribution holds in substance. (LLM, conf=0.85)
- Not matchedcites [9], [13]
A range of references, not one source.
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Mostly ICL exemplar selection typically relies on point-wise similarity [17]-[19], [66], [67], where a matching score is computed between each candidate exemplar and the input query, followed by the independent selection of the topk exemplars to construct the context [16].”
What the sources say
- Not matchedcites [17]-[19]
A range of references, not one source.
- Not matchedcites [66], [67]
A range of references, not one source.
- Supportedcites [16] [16] M. Luo, X. Xu, Y. Liu, P. Pasupat, and M. Kazemi, 'In-context learning with retrieved demonstrations for language models: A survey,' Transactions on Machine Learning Research , 2024. Survey Certification. - source
Section 3.1 states that of the two primary retrieval objectives, 'Most studies focus on language similarity' between demonstration and query, and Section 4 describes the typical retriever as computing a similarity measure (e.g. cosine) between each candidate demonstration embedding and the query embedding. Section 3.2 characterises the dominant/'most basic' inference strategy — one-hot retrieval — as ranking candidates by a scoring criterion and taking the top-k, so that 'each demonstration is chosen independently of the others', with clustering and iterative retrieval presented precisely as remedies for that point-wise, independent selection. Together these support the paraphrase, including its 'most/typically' quantifier. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Although these methods are straightforward and fast, they fail to consider compositional relationships among the selected k exemplars, often leading to redundancy caused by homogeneity.”
What the sources say
- Partly supportedcites [16] [16] M. Luo, X. Xu, Y. Liu, P. Pasupat, and M. Kazemi, 'In-context learning with retrieved demonstrations for language models: A survey,' Transactions on Machine Learning Research , 2024. Survey Certification. - sourceExperimental — accuracy not yet measured
Section 3.2 (Inference Strategy to Retrieve Few-shots Demonstrations) supports the substance: in one-hot retrieval 'each demonstration is chosen independently of the others' and the method 'is straightforward and fast, however, it might not yield the best combination of k demonstrations as these demonstrations might be homogeneous'; Section 3.1 likewise lists 'avoiding repetitive demonstrations' as a motivation for diversity. However, the source hedges the consequence ('might not yield the best combination', 'might be homogeneous'), whereas the citing sentence asserts it as a frequent outcome ('often leading to redundancy caused by homogeneity'), and the source nowhere uses 'redundancy' or 'compositional relationships' in this passage — that stronger causal/frequency claim is not stated by our copy. (LLM, conf=0.70)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Partly supported
67 more claims
Create a free account to see all 161 claims and to verify the 12 references still open.
- It may state two separate claims as one. 64
- Probably not a substantial claim. 32
Citations
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.09388 · Cited by 17,283
Self-citationLLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Simple Fast Algorithms for the Editing Distance between Trees and Related Problems
Zhang, K., & Shasha, D. (1989) · SIAM Journal on Computing · doi:10.1137/0218082 · Cited by 1,310
Emergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264 · cited 1×
“Recent studies have shown that LLMs exhibit strong ICL abilities as model and data sizes scale up [75], [76], [77].”
DECKARD: Scalable and Accurate Tree-Based Detection of Code Clones
Jiang, L., Misherghi, G., Su, Z., & Glondu, S. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.30 · Cited by 1,039
code2vec: Learning distributed representations of code
Alon, U., Zilberstein, M., Levy, O., & Yahav, E. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3290353 · Cited by 962
Unsupervised Machine Translation Using Monolingual Corpora Only
Lample, G., Conneau, A., Denoyer, L., & Ranzato, M. (2017) · arXiv · doi:10.48550/arxiv.1711.00043 · Cited by 584 · cited 1×
“Due to the success of deep learning, deep code translation methods have extended the SOTA further [57], [58], [59], [60], [61], [62].”
What Makes Good In-Context Examples for GPT-3?
Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., & Chen, W. (2022) · Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures · doi:10.18653/v1/2022.deelio-1.10 · Cited by 563
Best Algorithms for Approximating the Maximum of a Submodular Set Function
Nemhauser, G. L., & Wolsey, L. A. (1978) · Mathematics of Operations Research · doi:10.1287/moor.3.3.177 · Cited by 538
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C., Drain, D., Jiang, D., Tang, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.04664 · Cited by 536 · cited 2×
“We also extend our evaluation beyond code translation to code summarization using benchmarks from CodeXGLUE [36].”
Code Llama: Open Foundation Models for Code
Rozière, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.12950 · Cited by 531 · cited 2×
“Recently, large language models (LLMs) have demonstrated exceptional performance across various software mining tasks [1], [2], [3], [4], [5], [6], [7].”
Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.21783 · Cited by 505
GraphCodeBERT: Pre-training Code Representations with Data Flow
Guo, D., Ren, S., Lu, S., Feng, Z., Tang, D., Liu, S., Zhou, L., Duan, N., Svyatkovskiy, A., Fu, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2009.08366 · Cited by 480
Learning To Retrieve Prompts for In-Context Learning
Rubin, O., Herzig, J., & Berant, J. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.191 · Cited by 374
Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code
Pan, R., Ibrahimzada, A. R., Krishna, R., Sankar, D., Wassi, L. P., Merler, M., Sobolev, B., Pavuluri, R., Sinha, S., & Jabbarvand, R. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3639226 · Cited by 349
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
Nijkamp, E., Pang, B., Hayashi, H., Tu, L., Wang, H., Zhou, Y., Savarese, S., & Xiong, C. (2022) · arXiv · doi:10.48550/arxiv.2203.13474 · Cited by 330
Shaping program repair space with existing patches and similar code
Jiang, J., Xiong, Y., Zhang, H., Gao, Q., & Chen, X. (2018) · Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3213846.3213871 · Cited by 329 · cited 2×
“Mostly ICL exemplar selection typically relies on point-wise similarity [17], [18], [19], [66], [67], where a matching score is computed between each candidate exemplar and the input query, followed by the independent selection of the top-k exemplars to construct the context [16].”
StarCoder: may the source be with you!
Li, R., Allal, L. B., Zi, Y., Muennighoff, N., Kocetkov, D., Mou, C., Marone, M., Akiki, C., Li, J., Chim, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.06161 · Cited by 261
Learning to Generate Pseudo-Code from Source Code Using Statistical Machine Translation
Oda, Y., Fudaba, H., Neubig, G., Hata, H., Sakti, S., Toda, T., & Nakamura, S. (2015) · 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2015.36 · Cited by 250
Unified Demonstration Retriever for In-Context Learning
Li, X., Lv, K., Yan, H., Lin, T., Zhu, W., Ni, Y., Xie, G., Wang, X., & Qiu, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.256 · Cited by 245
Liu, S., Ye, H., Xing, L., & Zou, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.06668 · Cited by 231
Unified Demonstration Retriever for In-Context LearningPreprint
Li, X., Lv, K., Yan, H., Lin, T., Zhu, W., Ni, Y., Xie, G.-T., Wang, X., & Qiu, X. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2305.04320 · Cited by 224
CodeBLEU: a Method for Automatic Evaluation of Code Synthesis
Ren, S., Guo, D., Lu, S., Zhou, L., Liu, S., Tang, D., Sundaresan, N., Zhou, M., Blanco, A., & Ma, S. (2020) · arXiv · doi:10.48550/arxiv.2009.10297 · Cited by 214 · cited 1×
“In addition to computation accuracy, we also computed a range of metrics, including BLEU, Weighted N-Gram (WNG) Match Score, Syntactic Match Score, Dataflow Match Score, and CodeBLEU (weighted combination of the above four parts) [46], offering a more comprehensive evaluation of translation quality.”
Diverse Demonstrations Improve In-context Compositional GeneralizationPreprint
Levy, I., Bogin, B., & Berant, J. (2022) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2212.06800 · Cited by 211
DeepSeek-Coder: When the Large Language Model Meets Programming - The Rise of Code Intelligence
Guo, D., Zhu, Q., Yang, D., Xie, Z., Dong, K., Zhang, W., Chen, G., Bi, X., Wu, Y., Li, Y. K., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.14196 · Cited by 180
Tree-to-tree neural networks for program translation
Chen, X., Liu, C., & Song, D. (2018) · arXiv · doi:10.48550/arxiv.1802.03691 · Cited by 157
A Survey of Machine Learning for Big Code and Naturalness
Allamanis, M., Barr, E. T., Devanbu, P., & Sutton, C. (2019) · ACM Computing Surveys · doi:10.1145/3212695 · Cited by 150 · cited 1×
“Furthermore, phrase-based statistical machine translation techniques have shown notable advances [51], [52], [53], [54], [55], [56].”
Gemma 2: Improving Open Language Models at a Practical Size
Gemma Team, Riviere, M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., Ramé, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2408.00118 · Cited by 149
What Makes Good In-Context Examples for GPT-$3$?
Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2101.06804 · Cited by 148
Lexical statistical machine translation for language migration
Nguyen, A. T., Nguyen, T. T., & Nguyen, T. N. (2013) · Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering · doi:10.1145/2491411.2494584 · Cited by 136
Perfection Not Required? Human-AI Partnerships in Code Translation
Weisz, J. D., Muller, M., Houde, S., Richards, J., Ross, S. I., Martinez, F., Agarwal, M., & Talamadupula, K. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450656 · Cited by 129
Qwen, :, Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., et al. (2024) · arXiv · doi:10.48550/arxiv.2412.15115 · Cited by 128
Self-citationCompositional Exemplars for In-context LearningPreprint
Ye, J., Wu, Z., Feng, J., Yu, T., & Kong, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.05698 · Cited by 127
Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
Wang, B., Min, S., Deng, X., Shen, J., Wu, Y., Zettlemoyer, L., & Sun, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.153 · Cited by 120
Self-citationPhrase-Based Statistical Translation of Programming Languages
Karaivanov, S., Raychev, V., & Vechev, M. (2014) · Proceedings of the 2014 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming & Software · doi:10.1145/2661136.2661148 · Cited by 114
Self-Prompting Large Language Models for Zero-Shot Open-Domain QA
Li, J., Wang, J., Zhang, Z., & Zhao, H. Y. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.17 · Cited by 114
(2000) · International Journal of Productivity and Performance Management · doi:10.1108/ws.2000.07949fab.004 · Cited by 111
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes
Garg, S., Tsipras, D., Liang, P., & Valiant, G. (2022) · arXiv · doi:10.48550/arxiv.2208.01066 · Cited by 110
Exploring and Unleashing the Power of Large Language Models in Automated Code Translation
Yang, Z., Liu, F., Yu, Z., Keung, J. W., Li, J., Liu, S., Hong, Y., Ma, X., Jin, Z., & Li, G. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3660778 · Cited by 104
ChatGPT vs SBST: A Comparative Assessment of Unit Test Suite Generation
Tang, Y., Liu, Z., Zhou, Z., & Luo, X. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3382365 · Cited by 95
ATOM: Commit Message Generation Based on Abstract Syntax Tree and Hybrid Ranking
Liu, S., Gao, C., Chen, S., Nie, L. Y., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.3038681 · Cited by 93
Transform-Retrieve-Generate: Natural Language-Centric Outside-Knowledge Visual Question Answering
Gao, F., Ping, Q., Thattai, G., Reganti, A., Wu, Y. N., & Natarajan, P. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00501 · Cited by 92
Jana, P. K., Jha, P., Ju, H., Kishore, G., Mahajan, A., & Ganesh, V. (2024) · Frontiers in artificial intelligence and applications · doi:10.3233/faia240968 · Cited by 90
Transformers learn in-context by gradient descent
von Oswald, J., Niklasson, E., Randazzo, E., Sacramento, J., Mordvintsev, A., Zhmoginov, A., & Vladymyrov, M. (2022) · arXiv · doi:10.48550/arxiv.2212.07677 · Cited by 87 · cited 1×
“Additionally, several studies aim to provide theoretical justifications and insights into how LLMs learn from a few in-context demonstration exemplars [82], [83].”
Divide-and-Conquer Approach for Multi-phase Statistical Migration for Source Code (T)
Nguyen, A. T., Nguyen, T. T., & Nguyen, T. N. (2015) · 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2015.74 · Cited by 78
Mishra, B. D., Tafjord, O., & Clark, P. E. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.644 · Cited by 74
Diverse Demonstrations Improve In-context Compositional Generalization
Levy, I., Bogin, B., & Berant, J. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.78 · Cited by 66
What Can Transformers Learn In-Context? a Case Study of Simple Function Classes
Garg, S., Tsipras, D., Liang, P., & Valiant, G. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2217 · Cited by 65
Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., Huang, F., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.10671 · Cited by 63
Self-citationShi, P., Zhang, R., Bai, R. H., & Lin, J. (2022) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2210.13693 · Cited by 61
Mdp 3 : a Training-Free Approach for List-Wise Frame Selection in Video-Llms
Sun, H., Lu, S., Wang, H., Chen, Q.-G., Xu, Z., Luo, W., Zhang, K., & Li, M. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02233 · Cited by 61
Self-citationWeakly-Supervised Visual-Retriever-Reader for Knowledge-based Question Answering
Luo, M., Zeng, Y., Banerjee, P., & Baral, C. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.517 · Cited by 58
Unsupervised translation of programming languages
Roziere, B. (2026) · doi:10.70675/416f6582z4576z444eza710z5e930d59bf97 · Cited by 57
In-context Examples Selection for Machine Translation
Agrawal, S., Zhou, C., Lewis, M., Zettlemoyer, L., & Ghazvininejad, M. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.564 · Cited by 53
DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
DeepSeek-AI, Zhu, Q., Guo, D., Shao, Z., Yang, D., Wang, P., Xu, R., Wu, Y., Li, Y., Gao, H., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.11931 · Cited by 51 · cited 1×
“• DeepSeek-Coder [1], [40]: A series of open-source code LLMs, pre-trained on 2 trillion tokens using the fill-in-the-blank task to enhance code generation and infilling.”
Mishra, B. D., Tafjord, O., & Clark, P. (2022) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2204.13074 · Cited by 50
Macedo, M., Tian, Y., Cogo, F. R., & Adams, B. (2024) · IEEE/ACM International Conference on AI Foundation Models and Software Engineering (FORGE) · doi:10.1145/3650105.3652301 · Cited by 46
AVATAR: A Parallel Corpus for Java-Python Program Translation
Ahmad, W. U., Tushar, M. G. R., Chakraborty, S., & Chang, K.-W. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.143 · Cited by 46
Mapping API elements for code migration with vector representations
Nguyen, T. D., Nguyen, A. T., & Nguyen, T. N. (2016) · Proceedings of the 38th International Conference on Software Engineering Companion · doi:10.1145/2889160.2892661 · Cited by 46
AVATAR: A Parallel Corpus for Java-Python Program Translation
Ahmad, W. U., Tushar, M. G. R., Chakraborty, S., & Chang, K.-W. (2021) · arXiv · doi:10.48550/arxiv.2108.11590 · Cited by 46 · cited 2×
“Restrictions apply. approach, we evaluate various methods on AVATAR [35], a more realistic benchmark constructed from competitive programming sites, online platforms, and open-source repositories.”
Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning
Lu, P., Qiu, L., Chang, K.-W., Wu, Y. N., Zhu, S.-C., Rajpurohit, T., Clark, P., & Kalyan, A. (2022) · arXiv · doi:10.48550/arxiv.2209.14610 · Cited by 45 · cited 1×
“Recently, numerous studies have attempted to leverage ICL for complex tasks, including natural language understanding [17], reasoning [78], [79], [80], text generation [25], [26], [81].”
TreeBERT: A Tree-Based Pre-Trained Model for Programming LanguagePreprint
Jiang, X., Zheng, Z., Lyu, C., Li, L., & Lyu, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.12485 · Cited by 43
Multilingual Code Snippets Training for Program Translation
Zhu, M., Suresh, K., & Reddy, C. K. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21434 · Cited by 39 · cited 1×
“Due to the success of deep learning, deep code translation methods have extended the SOTA further [57], [58], [59], [60], [61], [62].”
Leveraging automated unit tests for unsupervised code translation
Roziere, B., Zhang, J. M., Charton, F., Harman, M., Synnaeve, G., & Lample, G. (2021) · arXiv · doi:10.48550/arxiv.2110.06773 · Cited by 39
RetICL: Sequential Retrieval of In-Context Examples with Reinforcement LearningPreprint
Scarlatos, A., & Lan, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.14502 · Cited by 37
UPRISE: Universal Prompt Retrieval for Improving Zero-Shot Evaluation
Cheng, D., Huang, S., Bi, J., Zhan, Y., Liu, J., Wang, Y., Sun, H., Wei, F., Deng, W., & Zhang, Q. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.758 · Cited by 32
Self-citationLearning Semantic Features for Software Defect Prediction by Code Comments Embedding
Huo, X., Yang, Y., Li, M., & Zhan, D. (2018) · Proceedings · doi:10.1109/icdm.2018.00133 · Cited by 32
Self-citationLong-context LLMs Struggle with Long In-context Learning
Li, T., Zhang, G., Do, Q. D., Yue, X., & Chen, W. (2024) · arXiv · doi:10.48550/arxiv.2404.02060 · Cited by 31
Code translation with compiler representations
Szafraniec, M., Roziere, B., Leather, H., Charton, F., Labatut, P., & Synnaeve, G. (2022) · arXiv · doi:10.48550/arxiv.2207.03578 · Cited by 26 · cited 5×
“It was released by [31] and is widely used due to multi-lingual samples and self-contained test suites [32], [33], [34].”
InfluentialBridging Gaps in LLM Code Translation: Reducing Errors with Call Graphs and Bridged Debuggers
Luo, Y., Yu, R., Zhang, F., Liang, L., & Xiong, Y. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695322 · Cited by 25
Shi, P., Zhang, R., Bai, H., & Lin, J. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.384 · Cited by 21
Pre-training Code Representation with Semantic Flow Graph for Effective Bug Localization
Du, Y., & Yu, Z. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3616338 · Cited by 19
Self-citationDeep Semantic Network Representation
Luo, X., Wu, J. X., Zhou, C. D., Zhang, X., & Wang, Y. (2020) · Proceedings · doi:10.1109/icdm50108.2020.00141 · Cited by 12
A Tool for Translating Sequential Source Code to Parallel Code Written in C++ and OpenACC
Alsubhi, K., Alsolami, F. J., Algarni, A., Albassam, E., Khemakhem, M. A., Eassa, F. E., Jambi, K. M., & Ashraf, M. U. (2019) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa47632.2019.9035292 · Cited by 12
Beyond Lexical Consistency: Preserving Semantic Consistency for Program Translation
Du, Y., Ma, Y.-F., Xie, Z., & Li, M. (2023) · Proceedings · doi:10.1109/icdm58522.2023.00018 · Cited by 12
Self-citationExploring translation of OpenMP to OpenACC 2.5: lessons learned
Pino, S., Pollock, L., & Chandrasekaran, S. (2017) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw.2017.84 · Cited by 11
In-context learning with retrieved demonstrations for language models: A survey
Luo, M., Xu, X., Liu, Y., Pasupat, P., & Kazemi, M. (2024) · arXiv · doi:10.48550/arxiv.2401.11624 · Cited by 9
Output format biases in the evaluation of large language models for code translation
Macedo, M., Tian, Y., Cogo, F. R., & Adams, B. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10768-1 · Cited by 7
AST-T5: Structure-aware pretraining for code generation and understanding
Gong, L., Elhoushi, M., & Cheung, A. (2024) · arXiv · doi:10.48550/arxiv.2401.03003 · Cited by 7 · cited 2×
“Previous studies have shown that explicitly incorporating code syntactic knowledge can significantly enhance performance in software mining tasks [8], [9], [10], [11], [12], [13], [14], [15].”
Enhancing LLMs in Long Code Translation through Instrumentation and Program State AlignmentPreprint
Li, X., Ya-Li, D., & Ming, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.02017 · Cited by 6
Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code
Pan, R., Ibrahimzada, A. R., Krishna, R., Sankar, D., Wassi, L. P., Merler, M., Sobolev, B., Pavuluri, R., Sinha, S., & Jabbarvand, R. (2023) · arXiv · Cited by 6
Let’s learn step by step: Enhancing in-context learning ability with curriculum learning
Liu, Y., Liu, J., Shi, X., Cheng, Q., Huang, Y., & Lu, W. (2024) · arXiv · doi:10.48550/arxiv.2402.10738 · Cited by 4 · cited 1×
“Despite significant progress, ICL still faces challenges and requires further exploration. such as its sensitivity to the selection of demonstration exemplars [84].”
Ma, Y.-F., Du, Y., & Li, M. (2023) · Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/249 · Cited by 4
Self-citationProgramming Language Translator For Integration Client Application With Web APIs
Tauda, M., Zainuddin, Z., & Tahir, Z. (2021) · IEEE International Conference on Artificial Intelligence and Mechatronics Systems (AIMS) · doi:10.1109/aims52415.2021.9466033 · Cited by 1
MDP3: A training-free approach for list-wise frame selection in video-LLMs
Sun, H., Lu, S., Wang, H., Chen, Q.-G., Xu, Z., Luo, W., Zhang, K., & Li, M. (2025) · arXiv · doi:10.48550/arxiv.2501.02885
Self-citationPyramid attention for source code summarization
Chai, L., & Li, M. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1485 · cited 2×
“In fact, mainstream methods based on LLMs often overlook code structural knowledge, whose effectiveness has been validated in prior studies [11], [12], [65].”
Self-citationOpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
Automatic inference of Java-to-swift translation rules for porting mobile applications
An, K., Meng, N., & Tilevich, E. (2018) · Proceedings of the 5th International Conference on Mobile Software Engineering and Systems · doi:10.1145/3197231.3197240
DeepSeek-Coder: When the large language model meets programming–the Rise of code intelligence
Guo (2024)
UnresolvedCode Llama: Open foundation models for code
Roziere (2023)
UnresolvedRevisiting chain-of-thought in code generation: Do language models need to learn reasoning before coding?
Liu (2025)
UnresolvedEnhancing LLMs in long code translation through instrumentation and program state alignment
Li (2025)
UnresolvedLong-context LLMs struggle with long in-context learning
Li (2024)
UnresolvedMoT: Pre-thinking and recalling enable ChatGPT to self-improve with memory-of-thoughts
Li (2023) · CoRR
UnresolvedGemma 2: Improving open language models at a practical size
Team (2024)
UnresolvedDeepSeek-Coder-V2: Breaking the barrier of closed-source models in code intelligence
Zhu (2024)
UnresolvedLlama: Open and efficient foundation language models
Touvron (2023)
UnresolvedLlama 2: Open foundation and fine-tuned chat models
Touvron (2023)
UnresolvedGPT-4 technical report
Achiam (2023)
UnresolvedCodeBLEU: A method for automatic evaluation of code synthesis
Ren (2020)
UnresolvedBabelTower: Learning to auto-parallelized program translation
Wen (2022)
UnresolvedA joint learning model with variational interaction for multilingual program translation
Du (2024)
UnresolvedChatGPT: Optimizing language models for dialogue
(2023)
UnresolvedThe Llama 3 herd of models
Dubey (2024)
UnresolvedIntroducing the next generation of Claude
(2024)
UnresolvedIn-context vectors: Making in context learning more effective and controllable through latent space steering
Liu (2023)
UnresolvedRetICL: Sequential retrieval of in-context examples with reinforcement learning
Scarlatos (2023)
UnresolvedLet’s learn step by step: Enhancing in-context learning ability with curriculum learning
Liu (2024)
Unresolved
Cited by
A Systematic Literature Review on Neural Code Translation
Chen, X., Xue, J., Xie, X., Liang, C., & Ju, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.07425 · Cited by 19
“Du et al. [19] introduced CAST, which is designed to enhance the performance of LLMs in code translation tasks under the In-Context Learning (ICL) setting by incorporating code structural information to guide exemplar selection.”
ACES: Who Tests the Tests? Leave-One-Out AUC Consistency for Code GenerationPreprint
Sun, H., Zhang, Y.-J., Xie, Z., Liu, R.-B., Du, Y., Li, X.-Y., & Li, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.03922 · Cited by 4
“Large language models (LLMs) have demonstrated strong capabilities in code generation (Chen et al., 2021; Guo et al., 2024; Liu et al., 2025; Du et al., 2025), yet individual generations are not always correct.”
Self-citationCIT-CAD: Constraint Intent Tree-based CAD Code Generation and VerificationPreprint
Du, Y., Sun, H., Xi, S.-Z., & Li, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.07434 · Cited by 3
“Prompting and example-selection work provides a complementary form of guidance, showing that code structure can matter when constructing LLM inputs [30], [31].”
Self-citationDesign-Specification Tiling for ICL-based CAD Code GenerationPreprint
Du, Y., Xi, S.-Z., Sun, H., & Li, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.12712 · Cited by 1
“(1) Random Sampling (Random): k exemplars are uniformly sampled from the candidate pool, independent of the test input X test : (2) Retrieval by Levenshtein Distance (LDSIM): Leven-shtein distance (LD), also as edit distance, is a fundamental string metric that quantifies similarity by counting the…”
InfluentialSelf-citationJia, S., Du, Y., & Li, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.30953
Self-citation
Metadata
- Authors
- Du, Y., Sun, H., & Li, M.
- Affiliations
- Nanjing University (CN)
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2025-11
- Volume
- 51
- Issue
- 11
- Pages
- 3038-3055
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W4413967102
- Semantic Scholar
- 6f5656607575700c9334186df37f4a054cc76058
- Cited by
- 0
- References
- 86
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Software Engineering (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62076121)
- Major Program (JD) of Hubei Province (2023BAA024)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- program translation, in-context learning, abstract syntax tree, exemplar selection, information coverage, submodular maximization, greedy approximation, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.