CoChat

Self-Refine: Iterative Refinement with Self-Feedback

Open access

The DOI matches its DataCite record; 48 of the 54 references it cites were found in a registry or an index; 12 of the 22 sources its claims cite were checked for support.

Abstract

Like humans, large language models (LLMs) do not always generate the best output on their first try. Motivated by how humans refine their written text, we introduce Self-Refine, an approach for improving initial outputs from LLMs through iterative feedback and refinement. The main idea is to generate an initial output using an LLMs; then, the same LLMs provides feedback for its output and uses it to refine itself, iteratively. Self-Refine does not require any supervised training data, additional training, or reinforcement learning, and instead uses a single LLM as the generator, refiner, and feedback provider. We evaluate Self-Refine across 7 diverse tasks, ranging from dialog response generation to mathematical reasoning, using state-of-the-art (GPT-3.5, ChatGPT, and GPT-4) LLMs. Across all evaluated tasks, outputs generated with Self-Refine are preferred by humans and automatic metrics over those generated with the same LLM using conventional one-step generation, improving by ~20% absolute on average in task performance. Our work demonstrates that even state-of-the-art LLMs like GPT-4 can be further improved at test time using our simple, standalone approach.

Overview

CoChat Summary

TL;DR

SELF-REFINE improves large language model outputs by iteratively generating self-feedback and refining the same model’s output without supervised training, additional training, or reinforcement learning. Across 7 diverse tasks, it improves over one-step generation by ∼20% absolute on average, with especially large gains on preference-based tasks.

Key findings

  • SELF-REFINE consistently improved GPT-3.5, ChatGPT, and GPT-4 across all evaluated tasks. Examples include GPT-4 Dialogue Response Generation improving from 25.4 to 74.6 (+49.2), GPT-4 Code Optimization from 27.3 to 36.0 (+8.7), and GPT-4 Constrained Generation from 15.0 to 45.0 (+30.0).
  • Feedback quality matters: in Code Optimization, SELF-REFINE feedback scored 27.5 vs. 26.0 for generic feedback and 24.8 for no feedback; in Sentiment Reversal, 43.2 vs. 31.2 vs. 0; in Acronym Generation, 56.4 vs. 54.0 vs. 48.0.
  • Iterative refinement generally improves outputs, with diminishing returns: Code Optimization rose from 22.0 at y0 to 28.8 at y3; Sentiment Reversal from 33.9 to 36.8; Constrained Generation from 29.0 to 49.7.
  • Math Reasoning gains were modest: GPT-3.5 64.1 to 64.1 (0), ChatGPT 74.8 to 75.0 (+0.2), GPT-4 92.9 to 93.1 (+0.2). With oracle feedback, GPT-3 improved by 4.8% and GPT-4 by 0.7%.

Methods

  • SELF-REFINE alternates between FEEDBACK and REFINE using the same model M: initial output y0 = M(pgen ∥ x); feedback fbt = M(pfb ∥ x ∥ yt); refinement yt+1 = M(prefine ∥ x ∥ y0 ∥ fb0 ∥ … ∥ yt ∥ fbt). It iterates until a stopping condition or up to a maximum of 4 iterations, using few-shot prompts and no supervised training, additional training, or reinforcement learning.
  • Evaluation covered 7 tasks: Dialogue Response Generation, Code Optimization, Code Readability Improvement, Math Reasoning, Sentiment Reversal, Acronym Generation, and Constrained Generation (CommonGen-Hard). Base LLMs were GPT-3.5 (text-davinci-003), ChatGPT (gpt-3.5-turbo), GPT-4, and CODEX for code tasks. Metrics included task-specific metrics, human preference, and GPT-4 preference.
  • Analysis included ablations on feedback quality and iteration count, comparison against generating multiple outputs, testing Vicuna-13B, and a qualitative error analysis of 70 samples for Code Optimization and Math Reasoning.

Limitations

  • Stated: the base models need sufficient few-shot modeling or instruction-following abilities to provide feedback and refine in-context; Vicuna-13B struggled to generate feedback in the required format and often failed to refine properly.
  • Stated: experiments used non-open-sourced models (GPT-3.5, ChatGPT, GPT-4, CODEX), whose pretraining corpus, sizes, and biases are not fully described, and which are not free to use; datasets were exclusively English; the approach does not explicitly guard against bad actors steering models toward toxic or harmful text.
  • Evident from the text: human A/B evaluations were conducted by the authors, and GPT-4 was used as a proxy for human preference with reported correlations of 82% for Sentiment Reversal, 68% for Acronym Generation, and 71% for Dialogue Response Generation. The paper also notes Math Reasoning gains are limited by the model’s inability to accurately identify errors, with ChatGPT feedback being “everything looks good” for 94% of instances.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2303.17651

Cites

48 of 54 references verified0 citation errors · 6 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Given an input ( 0 ⃝ ), SELF-REFINE starts by generating an output and passing it back to the same model M to get feedback ( 1 ⃝ ). The feedback is passed back to M , which refines the previously generated output ( 2 ⃝ ). Steps ( 1 ⃝ ) and ( 2 ⃝ ) iterate until a stopping condition is met. SELF-REFINE is instantiated with a language model such as GPT-3.5 and does not involve human assistance.
Figure 1 Given an input ( 0 ⃝ ), SELF-REFINE starts by generating an output and passing it back to the same model M to get feedback ( 1 ⃝ ). The feedback is passed back to M , which refines the previously generated output ( 2 ⃝ ). Steps ( 1 ⃝ ) and ( 2 ⃝ ) iterate until a stopping condition is met. SELF-REFINE is instantiated with a language model such as GPT-3.5 and does not involve human assistance.CC-BY · From the PDF
Figure 2 Examples of SELF-REFINE: an initial output generated by the base LLM and then passed back to the same LLM to receive feedback to the same LLM to refine the output . The top row illustrates this for dialog generation where an initial dialogue response can be transformed into a more engaging one that also understands the user by applying feedback. The bottom row illustrates this for code optimization where the code is made more efficient by applying feedback.
Figure 2 Examples of SELF-REFINE: an initial output generated by the base LLM and then passed back to the same LLM to receive feedback to the same LLM to refine the output . The top row illustrates this for dialog generation where an initial dialogue response can be transformed into a more engaging one that also understands the user by applying feedback. The bottom row illustrates this for code optimization where the code is made more efficient by applying feedback.CC-BY · From the PDF
Figure 4 Left : Iteration-wise score improvements. Early iterations significantly improve output quality, and scores generally keep improving with more iterations. Right : SELF-REFINE Performance improvements with iterations. Most gains( ∆ ) are in the initial iterations for both Code Opt. and Sentiment Reversal. The numbers are averaged over ChatGPT, GPT-3.5, and GPT-4. Task abbreviations: C. Opt. (Code Optimiz.), S. Rev. (Sentiment Reversal), C. Gen. (Constrained Generation).
Figure 4 Left : Iteration-wise score improvements. Early iterations significantly improve output quality, and scores generally keep improving with more iterations. Right : SELF-REFINE Performance improvements with iterations. Most gains( ∆ ) are in the initial iterations for both Code Opt. and Sentiment Reversal. The numbers are averaged over ChatGPT, GPT-3.5, and GPT-4. Task abbreviations: C. Opt. (Code Optimiz.), S. Rev. (Sentiment Reversal), C. Gen. (Constrained Generation).CC-BY · From the PDF
See Table 5 for an additional detailed comparison of related work.
See Table 5 for an additional detailed comparison of related work.CC-BY · From the PDF
Figure 6 Preference for the outputs generated by our method ( SELF-REFINE ), the multiple-sample baseline ( MULTI ), and ties ( ties ).
Figure 6 Preference for the outputs generated by our method ( SELF-REFINE ), the multiple-sample baseline ( MULTI ), and ties ( ties ).CC-BY · From the PDF
Figure 7 Initial web layout generated by our model for a fictional ice cream parlor.
Figure 7 Initial web layout generated by our model for a fictional ice cream parlor.CC-BY · From the PDF
Figure 8 Refined web layout after applying model feedback. The feedback included changing the background color to light blue (#6f2ff), increasing the heading font size to 48px, adding an icon before the welcome text, enhancing the content with an additional paragraph, increasing the button text size to 24px, and updating the button color to #9933.
Figure 8 Refined web layout after applying model feedback. The feedback included changing the background color to light blue (#6f2ff), increasing the heading font size to 48px, adding an icon before the welcome text, enhancing the content with an additional paragraph, increasing the button text size to 24px, and updating the button color to #9933.CC-BY · From the PDF
Figure 9 Initial web layout generated by our model for a page on photosynthesis.
Figure 9 Initial web layout generated by our model for a page on photosynthesis.CC-BY · From the PDF
Figure 10 Refined web layout after applying model feedback. The feedback included increasing the text font size to 18px for better readability, adding more information about the benefits of photosynthesis, removing the unnecessary margin-top from the header, and adding a ruler or divider below the header to separate it from the image.
Figure 10 Refined web layout after applying model feedback. The feedback included increasing the text font size to 18px for better readability, adding more information about the benefits of photosynthesis, removing the unnecessary margin-top from the header, and adding a ruler or divider below the header to separate it from the image.CC-BY · From the PDF
(a) Meaningful variable ratio across different SELF-REFINE iterations. (b) Comment per line ratio across different SELF-REFINE iterations. (c) Number of function units across different SELF-REFINE iterations.
(a) Meaningful variable ratio across different SELF-REFINE iterations. (b) Comment per line ratio across different SELF-REFINE iterations. (c) Number of function units across different SELF-REFINE iterations.CC-BY · From the PDF
Figure 13 SELF-REFINE prompts for dialogue response generation: INIT generates a first draft of the response generated in a few-shot manner. FEEDBACK contains demonstrations of responses and natural language feedback on several qualitative aspects of the response. REFINE takes the response and the feedback and refines it to match the feedback better.
Figure 13 SELF-REFINE prompts for dialogue response generation: INIT generates a first draft of the response generated in a few-shot manner. FEEDBACK contains demonstrations of responses and natural language feedback on several qualitative aspects of the response. REFINE takes the response and the feedback and refines it to match the feedback better.CC-BY · From the PDF
Figure 14 Improvements in accuracy on the GSM-8k math reasoning benchmark as a function of the # of iterations of SELF-REFINE.
Figure 14 Improvements in accuracy on the GSM-8k math reasoning benchmark as a function of the # of iterations of SELF-REFINE.CC-BY · From the PDF
Figure 15 A comparison of SELF-REFINE and direct generation with GPT-3.5 on CommonGenHard.
Figure 15 A comparison of SELF-REFINE and direct generation with GPT-3.5 on CommonGenHard.CC-BY · From the PDF
Figure 25 FEEDBACK prompt for Constrained Generation (truncated).
Figure 25 FEEDBACK prompt for Constrained Generation (truncated).CC-BY · From the PDF

Verification

Claims the paper makes

11 Valid · 2 Not valid · 4 Not checked · 107 Own results and interpretations

61 more claims

Create a free account to see all 124 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 25
  • It may state two separate claims as one. 35
  • The quote only partly states this claim. 5

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

65 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

5 of 9 · the rest aren’t loaded

4 more findings

Create a free account to see every finding of this check.

See all findings — free account

Citations

Cites

Show all 54 references

Research works printed but not linked: 3

  1. Sanjoy Dasgupta, Daniel Hsu, Stefanos Poulis, and Xiaojin Zhu. 2019. Teaching a black-box learner. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 1547-1555. PMLR.
  2. William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. 2022a. Self-critiquing models for assisting human evaluators.
  3. Timo Schick, Jane Dwivedi-Yu, Zhengbao Jiang, Fabio Petroni, Patrick Lewis, Gautier Izacard, Qingfei You, Christoforos Nalmpantis, Edouard Grave, and Sebastian Riedel. 2022a. Peer: A collaborative language model. · DOI · Paper page

Cited by

See all 2,314

Metadata

Authors
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., Gupta, S., Majumder, B. P., Hermann, K., Welleck, S., Yazdanbakhsh, A., & Clark, P.
Venue
arXiv
Publisher
arXiv
Published
2023
Pages
46534-46594
ISSN
1049-5258 (ISSN-L)
OpenAlex
W7133236450
Cited by
2,314
References
52
Citations by year
2026: 1329 · 2025: 460 · 2024: 338 · 2023: 186
FWCI
260.57
Type
conference paper
Language
en
Open access
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
Keywords
iterative refinement, self-feedback, large language models, mathematical reasoning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Self-Refine: Iterative Refinement with Self-Feedback. https://app.cochat.ai/papers/10.48550/arxiv.2303.17651
The paper
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., Gupta, S., Majumder, B. P., Hermann, K., Welleck, S., Yazdanbakhsh, A., & Clark, P. (2023). Self-Refine: Iterative Refinement with Self-Feedback. arXiv. https://doi.org/10.48550/arxiv.2303.17651

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.