Post-Incorporating Code Structural Knowledge Into Pretrained Models via ICL for Code Translation
Open accessThe DOI matches its Crossref record; 92 of the 112 references it cites were found in a registry or an index, and 15 are not checked yet; 7 of the 41 sources its claims cite were checked for support.
Abstract
Code translation migrates codebases across programming languages. Recently, large language models (LLMs) have achieved significant advancements in software mining. However, handling the syntactic structure of source code remains a challenge. Classic syntax-aware methods depend on intricate model architectures and loss functions, rendering their integration into LLM training resource-intensive. This paper employs in-context learning (ICL), which directly integrates task exemplars into the input context, to post-incorporate code structural knowledge into pre-trained LLMs. We revisit exemplar selection in ICL from an information-theoretic perspective, proposing that list-wise selection based on information coverage is more precise and general objective than traditional methods based on combine similarity and diversity. To address the challenges of quantifying information coverage, we introduce a surrogate measure, Coverage of Abstract Syntax Tree (CAST), measuring maximum subtree coverage between ASTs of test source code and exemplars. Furthermore, we formulate the NP-hard CAST maximization for exemplar selection and prove that it is a standard submodular maximization problem. Therefore, we propose a greedy algorithm for CAST submodular maximization, which theoretically guarantees a (1 − 1/e)-approximate solution in polynomial time complexity. Our method is the first training-free and model-agnostic approach to post-incorporate code structural knowledge into existing LLMs at test time. Experimental results show that our method significantly improves LLMs performance in code translation and reveals two meaningful insights: 1) Code structural knowledge can be effectively post-incorporated into pre-trained LLMs during inference, despite being overlooked during training; 2) Scaling up model size or training data does not lead to the emergence of code structural knowledge, underscoring the necessity of explicitly considering code syntactic structure.
Verified 2026-11-19 · 15 references not checked yet
Overview
CoChat Summary
## TL;DR
The paper proposes CAST, a training-free, model-agnostic in-context learning retrieval method that post-incorporates code structural knowledge into pretrained LLMs for code translation. Across code translation benchmarks, CAST-F generally outperforms similarity- and diversity-based exemplar selection, and the paper reports that scaling model size alone does not make structural knowledge emerge sufficiently.
## Key findings
- On 5-shot code translation with Qwen2-7B, CAST-F achieved an average CA of 64.28%, higher than Random (61.28%), LD (62.33%), AST-ED (63.12%), BM25 (62.09%), CodeBERT (61.91%), and Diversity (62.68%).
- For Gemma2-27B, CAST-F reached 70.03% average CA, exceeding Random (68.81%) and Diversity (68.13%); for Qwen2.5-72B, CAST-F reached 71.33% average CA versus Diversity at 70.19%.
- On AVATAR with Qwen2-7B, CAST-F achieved the best reported computational accuracy in the table: 47.79% for Java→Python and 50.40% for Python→Java, and the paper says it also gave the best overall CodeBLEU scores.
- For code summarization on CodeXGLUE, CAST-F achieved the highest BLEU and Rouge-L scores across nearly all languages and metrics; for example, on Java it scored 2.42 BLEU / 9.90 Rouge-L, and on Ruby 6.71 BLEU / 14.47 Rouge-L.
- In the failure analysis for Python→Java 20-shot translation, CAST-F had the lowest Wrong Answer rate (26.41%) and the highest Accept rate (58.10%).
## Methods
- The paper formulates code translation as ICL with exemplar selection and proposes CAST (Coverage of AST), a surrogate score for information coverage based on the ratio of maximum covered subtrees between the test code AST and exemplar ASTs.
- It uses tree-sitter to extract ASTs, keeps only node types, computes subtree fingerprints by hashing, builds a co-occurrence matrix between test subtrees and database samples, and selects exemplars by maximizing CAST.
- CAST maximization is shown to be NP-hard but a non-negative, monotone, submodular maximization problem; the paper uses a greedy algorithm with a theoretical (1 - 1/e) approximation guarantee and evaluates it on 948 parallel functions, focusing on the 568 functions with unit tests, plus AVATAR and CodeXGLUE code summarization benchmarks.
## Limitations
- The paper explicitly states that improving CAST may require considering more advanced code structure beyond AST, such as CFG and DFG, and suggests a feedback loop during inference as future work.
- The threat analysis states a possible internal validity threat from LLM data leakage due to training/testing overlap, though it says prior work suggests the threat is limited.
- It also notes an external validity threat from dataset quality and an external validity threat from implementation differences across compared techniques; these are mitigated by using state-of-the-art models and a cleaned, widely used dataset, but the threat itself is still stated.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
6 Valid · 8 Not valid · 21 Not checked · 126 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“explicitly incorporating structural knowledge into models enhances their perceptual understanding of code, thereby aiding software mining tasks [14], [15], including code translation [9], [13].”
What the sources say
- Supportedcites [14] source
The Motivation (2.2, p. 3) states that program comprehension tasks — vulnerability identification, function name prediction, source code summarization — 'have utilized code structure such as Abstract Syntax Tree (AST) to learn code semantics and good performance has been demonstrated', and 2.3 (p. 3) adds that 'Adopting AST in code comprehension has been proved to get the state-of-the-art performance'. The Abstract (p. 1) and 5.1 (p. 11) show the paper itself explicitly incorporating AST because it 'can be important for capturing code semantics', yielding a 30.72% BLEU-4 gain on commit message generation, which the paper frames as NMT translation of code changes into language (6.1, p. 15; 6.2, p. 15 also notes Wan et al. incorporating an AST to translate code snippets). This broad background paraphrase is thus treated by our copy, at the level of generality the citing sentence uses. (LLM, conf=0.80)
- Supportedcites [15] Gong, 2024, AST-T5: Structure-aware pretraining for code generation and understanding source
AST-T5's Abstract and 5.2 Main Results state that structure-awareness aids both code understanding and code-to-code translation: 'The benefits of being structure-aware are further exemplified by AST-T5's leading results in Clone Detection, where it surpasses CodeT5 by 3 points' and 'Structure-awareness makes AST-T5 particularly powerful in code-to-code tasks, surpassing CodeT5 by ... 3 points in exact match score for Java-C# Transpilation in CodeXGLUE.' The Introduction adds the general point that 'leveraging the Abstract Syntax Tree (AST) of code can significantly improve performance on code-related tasks,' and the ablation table (Section 5) shows AST-aware segmentation/corruption raising the Understanding columns. The paper does not use the phrase 'software mining,' but its evaluated tasks (clone detection on BigCloneBench, defect detection, transpilation) are exactly those, so the attribution holds in substance. (LLM, conf=0.85)
- Not matchedcites [9], [13]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Mostly ICL exemplar selection typically relies on point-wise similarity [17]-[19], [66], [67], where a matching score is computed between each candidate exemplar and the input query, followed by the independent selection of the topk exemplars to construct the context [16].”
What the sources say
- Not matchedcites [17]-[19]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites [66], [67]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites [16] Luo, 2024, In-context learning with retrieved demonstrations for language models: A survey source
Section 3.1 states that of the two primary retrieval objectives, 'Most studies focus on language similarity' between demonstration and query, and Section 4 describes the typical retriever as computing a similarity measure (e.g. cosine) between each candidate demonstration embedding and the query embedding. Section 3.2 characterises the dominant/'most basic' inference strategy — one-hot retrieval — as ranking candidates by a scoring criterion and taking the top-k, so that 'each demonstration is chosen independently of the others', with clustering and iterative retrieval presented precisely as remedies for that point-wise, independent selection. Together these support the paraphrase, including its 'most/typically' quantifier. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Although these methods are straightforward and fast, they fail to consider compositional relationships among the selected k exemplars, often leading to redundancy caused by homogeneity.”
What the sources say
- Partly supportedcites [16] Luo, 2024, In-context learning with retrieved demonstrations for language models: A survey sourceExperimental — accuracy not yet measured
Section 3.2 (Inference Strategy to Retrieve Few-shots Demonstrations) supports the substance: in one-hot retrieval 'each demonstration is chosen independently of the others' and the method 'is straightforward and fast, however, it might not yield the best combination of k demonstrations as these demonstrations might be homogeneous'; Section 3.1 likewise lists 'avoiding repetitive demonstrations' as a motivation for diversity. However, the source hedges the consequence ('might not yield the best combination', 'might be homogeneous'), whereas the citing sentence asserts it as a frequent outcome ('often leading to redundancy caused by homogeneity'), and the source nowhere uses 'redundancy' or 'compositional relationships' in this passage — that stronger causal/frequency claim is not stated by our copy. (LLM, conf=0.70)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Partly supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In summary, constructing the co-occurrence matrix for CAST takes O ( n · | Φ test | ) , while the greedy CAST algorithm in Algorithm 2 has a polynomial time complexity of O ( nk · | Φ test | ) .”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“as well as the best overall CodeBLEU scores.”
plainn = 6,255- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
67 more claims
Create a free account to see all 161 claims and to verify the 14 references still open.
- It may state two separate claims as one. 64
- Probably not a substantial claim. 32
Citations
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.09388 · Cited by 17,283
Self-citationLLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590
Simple Fast Algorithms for the Editing Distance between Trees and Related Problems
Zhang, K., & Shasha, D. (1989) · SIAM Journal on Computing · doi:10.1137/0218082 · Cited by 1,310
Emergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264 · cited 1×
“Recent studies have shown that LLMs exhibit strong ICL abilities as model and data sizes scale up [75], [76], [77].”
DECKARD: Scalable and Accurate Tree-Based Detection of Code Clones
Jiang, L., Misherghi, G., Su, Z., & Glondu, S. (2007) · 29th International Conference on Software Engineering (ICSE'07) · doi:10.1109/icse.2007.30 · Cited by 1,039
code2vec: Learning distributed representations of code
Alon, U., Zilberstein, M., Levy, O., & Yahav, E. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3290353 · Cited by 962
Cited by
A Systematic Literature Review on Neural Code Translation
Chen, X., Xue, J., Xie, X., Liang, C., & Ju, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.07425 · Cited by 19
“Du et al. [19] introduced CAST, which is designed to enhance the performance of LLMs in code translation tasks under the In-Context Learning (ICL) setting by incorporating code structural information to guide exemplar selection.”
ACES: Who Tests the Tests? Leave-One-Out AUC Consistency for Code GenerationPreprint
Sun, H., Zhang, Y.-J., Xie, Z., Liu, R.-B., Du, Y., Li, X.-Y., & Li, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.03922 · Cited by 4
“Large language models (LLMs) have demonstrated strong capabilities in code generation (Chen et al., 2021; Guo et al., 2024; Liu et al., 2025; Du et al., 2025), yet individual generations are not always correct.”
Self-citationCIT-CAD: Constraint Intent Tree-based CAD Code Generation and VerificationPreprint
Du, Y., Sun, H., Xi, S.-Z., & Li, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.07434 · Cited by 3
“Prompting and example-selection work provides a complementary form of guidance, showing that code structure can matter when constructing LLM inputs [30], [31].”
Self-citationDesign-Specification Tiling for ICL-based CAD Code GenerationPreprint
Du, Y., Xi, S.-Z., Sun, H., & Li, M. (2026) · arXiv.org · doi:10.48550/arxiv.2603.12712 · Cited by 1
“(1) Random Sampling (Random): k exemplars are uniformly sampled from the candidate pool, independent of the test input X test : (2) Retrieval by Levenshtein Distance (LDSIM): Leven-shtein distance (LD), also as edit distance, is a fundamental string metric that quantifies similarity by counting the…”
InfluentialSelf-citationJia, S., Du, Y., & Li, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.30953
Self-citation
Metadata
- Authors
- Du, Y., Sun, H., & Li, M.
- Affiliations
- Nanjing University (CN)
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2025-11
- Volume
- 51
- Issue
- 11
- Pages
- 3038-3055
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W4413967102
- Semantic Scholar
- 6f5656607575700c9334186df37f4a054cc76058
- Cited by
- 0
- References
- 86
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Software Engineering (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (62076121)
- Major Program (JD) of Hubei Province (2023BAA024)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- program translation, in-context learning, abstract syntax tree, exemplar selection, information coverage, submodular maximization, greedy approximation, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Post-Incorporating Code Structural Knowledge Into Pretrained Models via ICL for Code Translation. https://app.cochat.ai/papers/10.1109/tse.2025.3605768
- The paper
- Du, Y., Sun, H., & Li, M. (2025). Post-Incorporating Code Structural Knowledge Into Pretrained Models via ICL for Code Translation. IEEE Transactions on Software Engineering. https://doi.org/10.1109/tse.2025.3605768
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.