CoChat
Sign Up
PreprintCited by 1Software Engineering ResearchTopic ModelingTeaching and Learning Programming

RefineCoder: Iterative Improving of Large Language Models via Adaptive Critique Refinement for Code Generation

Open access
Zhou, Changzhi, Zhang, Xinyu, Song, Dandan, Chen, Xiancai, Gu, Wanli, Ma, Huipeng, Tian, Yuhang, Zhang, Mengdi +1 more·2025·arXiv
DOI 10.48550/arxiv.2502.09183arXiv 2502.09183
Open this paper in CoChat — free accountOpen PDF

Abstract

Code generation has attracted increasing attention with the rise of Large Language Models (LLMs). Many studies have developed powerful code LLMs by synthesizing code-related instruction data and applying supervised fine-tuning. However, these methods are limited by teacher model distillation and ignore the potential of iterative refinement by self-generated code. In this paper, we propose Adaptive Critique Refinement (ACR), which enables the model to refine itself by self-generated code and external critique, rather than directly imitating the code responses of the teacher model. Concretely, ACR includes a composite scoring system with LLM-as-a-Judge to evaluate the quality of code responses and a selective critique strategy with LLM-as-a-Critic to critique self-generated low-quality code responses. We develop the RefineCoder series by iteratively applying ACR, achieving continuous performance improvement on multiple code generation benchmarks. Compared to the baselines of the same size, our proposed RefineCoder series can achieve comparable or even superior performance using less data.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper proposes Adaptive Critique Refinement (ACR), an iterative fine-tuning paradigm that refines code LLMs using self-generated code plus teacher-generated critique rather than directly imitating teacher code. The resulting RefineCoder models improve average Pass@1 over three iterations on multiple code benchmarks while using only 80K training examples.

Key findings

  • On LiveCodeBench and BigCodeBench-hard, RefineCoder-DS-6.7B and RefineCoder-QW-7B improve average Pass@1 by 2.7 and 3.1 points after three ACR iterations; iter0-to-iter1 gains are 2.0 and 2.3 points. The paper reports that RefineCoder-DS-6.7B and RefineCoder-QW-7B (Iter3) outperform same-size baselines by 3.6 and 2.3 points using only 80K data.
  • On HumanEval(+), MBPP(+), and BigCodeBench, three iterations yield average improvements of 2.7 and 4.5 points for the two RefineCoder models. RefineCoder-QW-7B Iter3 reaches an average of 70.4, while RefineCoder-DS-6.7B Iter3 reaches 64.0.
  • The composite scoring system is validated by sampling 10 code responses per benchmark question: highest-scoring code far exceeds lowest-scoring code, with low error rates. For example, RefineCoder-QW-7B Iter3 scores 83.6 best vs. 75.0 worst on HumanEval+ (error rate 3.1) and 73.4 best vs. 71.1 worst on MBPP+ (error rate 5.3).
  • Data leakage analysis reports TLI values for HumanEval+/MBPP+ of 43.20/19.40 for Magic-Evol-Instruct, 47.04/19.46 for Evol-CodeAlpaca-v1, and 30.50/17.67 for Code-FeedBack, versus 4.97/7.00 for the paper’s dataset. The paper states Magic-OSS-Instruct and its own dataset are entirely free from data leakage, while the other three datasets show severe leakage.

Methods

  • ACR iteratively samples 7 code responses per instruction at temperature 0.7, ranks them with a composite scoring system combining multi-faceted LLM-as-a-Judge criteria (Q&A consistency, code existence, correctness, clarity, comments) and an executor score, then uses pairwise judgment against the teacher-generated response to select a winner and loser.
  • A selective critiquing strategy uses LLM-as-a-Critic when the teacher response wins, producing two-turn data `(x, y_l1, c, y_w)`; otherwise it constructs single-turn data `(x, y_w)`. The model is fine-tuned from scratch each iteration to prevent overfitting.
  • Initial SFT dataset D0 contains 80K high-quality Python instructions built by combining Evol-Instruct and SelfCodeAlign with GPT-4o. Base models are DS-Coder-6.7B-Base and Qwen2.5-Coder-7B-Base; Qwen2.5-32B-Instruct serves as Judge and Critic. Training uses 2 epochs, global batch size 64, learning rate 5e-6, AdamW with cosine decay, and 16 A100-80G GPUs via LLaMA-Factory. Evaluation uses Pass@1 on HumanEval(+), MBPP(+), LiveCodeBench, BigCodeBench (hard), and MultiPL-E.

Limitations

  • The paper states ACR primarily refines code responses based on programming instructions, so it still requires an initial set of high-quality, diverse instructions and a specialized code instruction generator to become more automated. It also states ACR may apply to other reasoning-intensive tasks such as mathematics, but this paper does not fully explore those domains.
  • The paper reports diminishing returns after the third iteration, suggesting the model may enter a mild overfitting stage where further refinement fails to improve performance and may worsen it.
  • The paper notes that severe data leakage in several baseline datasets makes it challenging for RefineCoder to surpass them on HumanEval(+) and MBPP+, and RefineCoder still lags behind the state-of-the-art baseline on those two benchmarks.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2502.09183

Cites

49 of 57 references verified0 citation errors · 8 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

2 of 6 cited sources checked · 0 supported · 0 flagged · accuracy not yet measured

  • Findingown data

    Following three iterations, the two RefineCoder models improve average pass@1 substantially across HumanEval(+), MBPP(+), LiveCodeBench, and BigCodeBench (hard).

    “After three iterations, RefineCoder-DS-6.7B and RefineCoder-QW-7B demonstrate significant improvements in average pass@1 performance on the HumanEval(+), MBPP(+), LiveCodeBench, and BigCodeBench (hard)”
    Quote in the paperSubstantial claim
  • Findingown data

    Three datasets-Magic-Evol-Instruct, Evol-CodeAlpaca-v1, and Code-FeedBack-exhibit serious leakage with respect to HumanEval(+) and MBPP(+).

    “In contrast, Magic-Evol-Instruct, Evol-CodeAlpaca-v1, and Code-FeedBack demonstrate severe data leakage concerning HumanEval (+) and MBPP (+).”
    Quote in the paperSubstantial claim
  • Findingown data

    The proposed models attain the highest average Pass@1 among all compared baselines.

    “Compared with all baselines, our models achieve the best average Pass@1 performance.”
    Quote in the paperSubstantial claim
  • Findingown data

    Repeated application of ACR to the RefineCoder models produces ongoing gains on several code generation benchmarks.

    “We develop the RefineCoder series by iteratively applying ACR, achieving continuous performance improvement on multiple code generation benchmarks.”
    • Also stated in: § 1 Introduction · plain
      “By iteratively applying the ACR process, the model achieves continuous improvement in its code generation capabilities.”
    • Also stated in: § 1 Introduction · plain
      “Experimental results from the RefineCoder series show that iterative ACR continuously improves code generation performance.”
    Quote in the paperSubstantial claim
  • Findingown data

    The third-iteration RefineCoder-QW-7B obtains top results for half of the programming languages.

    “In particular, RefineCoder-QW-7B (Iter3) achieves the best results among half of the programming languages.”
    Quote in the paperSubstantial claim
  • May combine two claims. 21
  • The quote only partly states this claim. 1
  • Classified as probably not a substantial claim. 6

  • May combine two claims.

    Premisecitation

    Earlier methods evaluate code quality by generating test cases, but this suffers from inaccurate test generation, inability to handle special judges, and failure to assess non-code parts of responses.

    “Previous works (Chen et al., 2023;Zhang et al., 2024a) rely on generating test cases to evaluate code quality.However, this method has issues with inaccurate test case generation (Jain et al., 2024), inability to handle special judge (Quan et al., 2025), and failure to evaluate non-code content in response such as code comments and explanations.”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites Chen et al., 2023 source

      The citation to Chen et al. is attached to the claim that earlier work generates test cases to evaluate code quality. The paper’s “Execution-based code selection” section describes using unit tests supplied in the problem description to check code correctness, not generating those tests. The later limitations are attached to other citations, so they are not assessed as claims attributed to Chen et al. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites Zhang et al., 2024a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Jain et al., 2024 source

      The Jain et al. citation is attached to the concern about inaccurate test-case generation, not to the separate claims about special judges or non-code content. Section 3 describes insufficient tests in existing benchmarks, and Appendix A.2 says randomly sampled inputs may miss corner cases. These passages support a concern about test coverage, but do not establish that generated test cases are inaccurate. (LLM, conf=0.75)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingreasoning

    The data construction pipeline yields an 80,000-example Python instruction dataset that is high-quality and diverse.

    “Through this pipeline, we obtain an 80K highquality and diverse python instruction dataset.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: GPT-4o produces high-quality and diverse code-related concepts, programming instructions, and code responses for the initial SFT dataset.

  • May combine two claims.

    Findingown data

    At one and three iterations, the two RefineCoder models reach the best average MultiPL-E scores, exceeding their initial versions by 1.0 and 2.1 points, respectively.

    “After one and three iterations, RefineCoder-DS-6.7B and RefineCoder-QW-7B achieve optimal average performance, surpassing the initial model by 1.0 and 2.1 points, respectively.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    With human-like improvement suggestions, RefineCoder-QW-7B at iterations 1 and 3 surpasses GPT-4o on three benchmarks.

    “RefineCoder-QW-7B (Iter1 and Iter3) outperforms GPT-4o on three benchmarks”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The RefineCoder models outperform same-sized baselines while using a smaller amount of data.

    “Compared to the baselines of the same size, our proposed RefineCoder series can achieve superior performance using less data.”
    • Also stated in: § 1 Introduction · plain
      “After three iterations, the RefineCoder models outperform baselines of the same size while requiring less data.”
    • Also stated in: § 4.5 Experimental Results · plain
      “Compared to baselines of the same size, the RefineCoder-DS-6.7B and RefineCoder-QW-7B (Iter3) outperform them by 3.6 and 2.3 points, respectively, using only 80K data.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Against same-sized baselines, the third-iteration RefineCoder-DS-6.7B and RefineCoder-QW-7B exceed them by 3.6 and 2.3 points, respectively, with only 80K training examples.

    “Compared to baselines of the same size, the RefineCoder-DS-6.7B and RefineCoder-QW-7B (Iter3) outperform them by 3.6 and 2.3 points, respectively, using only 80K data.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Code selected as best performs much better than code selected as worst, and the scoring method has a low error rate.

    “As shown in Table 4, the performance of the highest-scoring code far exceeds that of the lowest-scoring code, while the scoring system maintains a low error rate.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    From iteration 0 to iteration 1, performance rises by 2.0 and 2.3 points.

    “Notably, the performance improved by 2.0 and 2.3 points from iter0 to iter1”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    RefineCoder-DS-6.7B at iteration 3 achieves an average score of 64.0 across HumanEval, MBPP, and BigCodeBench, which is 2.7 points higher than its iteration 0.

    “| RefineCoder-DS-6.7B Iter3 | 80K | | 75.0 70.7 80.2 67.2 41.1 | 49.6 | 64.0 ↑2.7 | | |”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

    Findingnone given

    The authors' dataset has lower TLI values on HumanEval+ and MBPP+ than the Magic-Evol-Instruct, Evol-CodeAlpaca-v1, and Code-FeedBack datasets.

    “| Ours | | | 80K | 4.97 | 7.00 | 1.54 | 5.00 |”
    • Also stated in: § Prompt for Instruction Evolution: Diversion · plain
      “| Magic-Evol-Instruct (Wei et al., 2024b) | 110K | 43.20 | 19.40 | 2.91 | 4.10 |”
    • Also stated in: § Prompt for Instruction Evolution: Diversion · plain
      “| Evol-CodeAlpaca-v1 (Luo et al., 2024) | 110K | 47.04 | 19.46 | 2.91 | 4.69 |”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    RefineCoder-QW-7B at iteration 2 achieves an average score of 69.6, an improvement of 3.7 points over iteration 0.

    “| RefineCoder-QW-7B Iter2 | 80K | | 86.6 82.5 83.4 71.7 42.3 | 50.9 | 69.6 ↑3.7 | | |”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The authors' dataset of 80K samples has a Test Leakage Indicator (TLI) of 4.97 on HumanEval+, 7.00 on MBPP+, 1.54 on LiveCodeBench, and 5.00 on BigCodeBench.

    “| Ours | | | 80K | 4.97 | 7.00 | 1.54 | 5.00 |”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    RefineCoder-DS-6.7B at iteration 2 achieves an average score of 63.3, an improvement of 2.0 points over iteration 0.

    “| RefineCoder-DS-6.7B Iter2 | 80K | | 74.4 70.3 79.6 66.9 40.6 | 47.9 | 63.3 ↑2.0 | | |”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    RefineCoder-QW-7B at iteration 1 achieves an average score of 68.5, an improvement of 2.6 points over iteration 0.

    “| RefineCoder-QW-7B Iter1 | 80K | | 86.0 79.3 84.1 70.8 41.6 | 49.0 | 68.5 ↑2.6 | | |”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    RefineCoder-DS-6.7B at iteration 1 achieves an average score of 62.6, an improvement of 1.3 points over iteration 0.

    “| RefineCoder-DS-6.7B Iter1 | 80K | | 74.4 68.9 77.0 66.4 40.6 | 48.0 | 62.6 ↑1.3 | | |”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    RefineCoder-QW-7B at iteration 3 achieves an average score of 70.4, an improvement of 4.5 points over iteration 0.

    “| RefineCoder-QW-7B Iter3 | 80K | | 87.2 83.3 85.6 72.9 42.0 | 51.1 | 70.4 ↑4.5 | | |”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The model might have begun mild overfitting, so additional refinement could stop helping and even worsen results.

    “This suggests that the model may have entered a mild overfitting stage, where further refinement not only fails to improve performance but may also exacerbate the issue.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    To make ACR more automatic, a dedicated code instruction generator remains necessary.

    “Therefore, a specialized code instruction generator is still required to make ACR more automated.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given

    ACR mainly refines code answers from programming instructions, so it requires an initial collection of high-quality, varied instructions.

    “First, ACR is primarily designed to refine code responses based on programming instructions, necessitating an initial set of high-quality and diverse instructions.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    ACR departs from standard teacher distillation and enhances inherent single-pass code generation capability.

    “which breaks away from the traditional teacher distillation paradigm and improves the intrinsic one-off code generation ability.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given

    The combined scoring approach using an LLM judge and the selective critique approach using an LLM critic allow precise evaluation and feedback on code responses.

    “To ensure the effectiveness of ACR, we design a composite scoring system with LLM-as-a-Judge and a selective critiquing strategy with LLM-as-a-Critic, enabling both accurate scoring and critique of code responses.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The teacher-generated code responses (y0) are of sufficiently high quality to serve as a reference for critique and as a potential winner.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Calling LLMs repeatedly raises inference latency.

    “Besides, multiple calls to LLMs increase inference latency.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    Parameter-frozen approaches with external feedback amount to prompt engineering and do not enhance an LLM's inherent single-pass code generation skill.

    “However, these methods that freeze parameters are essentially a form of prompt engineering with external feedback.They cannot improve the intrinsic one-off code generation ability of LLMs.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The model can recognize and fix weaknesses by learning from its own code and outside feedback.

    “This demonstrates that the model is capable of identifying and addressing deficiencies by learning from both its self-generated code and external feedback.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    In distillation-based fine-tuning, the student's performance is heavily tied to the teacher, and the possibility of refining with self-generated code is overlooked.

    “However, this paradigm of teacher model distillation inevitably suffers from a critical limitation: the performance of the student model largely relies on the teacher model, ignoring the potential of iterative refinement by self-generated code.”
    Quote in the paperNot a substantial claim
25 of 25

  • The paper relies on this without support.

    GPT-4o produces high-quality and diverse code-related concepts, programming instructions, and code responses for the initial SFT dataset.

    “we first prompt GPT-4o to generate code-related concepts and corresponding programming instructions from Python code snippets.”
    “Finally, we prompt GPT-4o to generate code responses for the instructions.”

  • The paper relies on this without support.

    The teacher-generated code responses (y0) are of sufficiently high quality to serve as a reference for critique and as a potential winner.

    “we determine the winner between the best self-generated code y self and the teacher-generated code y 0 based on the pairwise judgment and execution scores”
    “LLM-as-a-Critic is used to critique y l 1 , using y 0 as a reference”
2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Dongyoung Kim, Jaehyung Kim, Kimin Lee, and Jin- woo Shin. 2025. Spread preference annotation: Di- rect preference judgment for efficient LLM align- ment. In The Thirteenth International Conference on Learning Representations.

Cited by

Sorted by citations

Related

Authors
Zhou, C., Zhang, X., Song, D., Chen, X., Gu, W., Ma, H., Tian, Y., Zhang, M., & Hu, L.
Affiliations
  • Beijing Institute of Technology (CN)
  • Peking University (CN)
  • Meituan (China) (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2502.09183
arXiv
2502.09183
OpenAlex
W4407571908
Cited by
0
References
56
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
Keywords
code generation, large language models, iterative refinement, LLM-as-a-judge

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review