CoChat
Sign Up
PreprintCited by 10Topic ModelingNatural Language Processing TechniquesMultimodal Machine Learning Applications

Small Language Model Can Self-correct

Open access
Han, Haixia, Liang, Jiaqing, Shi, Jie, He, Qianyu, Xiao, Yanghua·2024·arXiv
DOI 10.48550/arxiv.2401.07301arXiv 2401.07301
Open this paper in CoChat — free accountOpen PDF

Abstract

Generative Language Models (LMs) such as ChatGPT have exhibited remarkable performance across various downstream tasks. Nevertheless, one of their most prominent drawbacks is generating inaccurate or false information with a confident tone. Previous studies have devised sophisticated pipelines and prompts to induce large LMs to exhibit the capability for self-correction. However, large LMs are explicitly prompted to verify and modify its answers separately rather than completing all steps spontaneously like humans. Moreover, these complex prompts are extremely challenging for small LMs to follow. In this paper, we introduce the \underline{I}ntrinsic \underline{S}elf-\underline{C}orrection (ISC) in generative language models, aiming to correct the initial output of LMs in a self-triggered manner, even for those small LMs with 6 billion parameters. Specifically, we devise a pipeline for constructing self-correction data and propose Partial Answer Masking (PAM), aiming to endow the model with the capability for intrinsic self-correction through fine-tuning. We conduct experiments using LMs with parameters sizes ranging from 6 billion to 13 billion in two tasks, including commonsense reasoning and factual knowledge reasoning. Our experiments demonstrate that the outputs generated using ISC outperform those generated without self-correction. We believe that the output quality of even small LMs can be further improved by empowering them with the ability to intrinsic self-correct.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2401.07301

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

  • Scaling Laws for Neural Language Models

    Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · Cited by 1,749 · cited 1× · see citing works

    “Scaling model size can increase model’s various capabilities(Kaplan et al. 2020), including self-verification (or self-feedback).”
    Background

Flagged first, then by citations

Research works printed but not linked: 26

  1. Azaria, A.; and Mitchell, T. 2023. The Internal State of an LLM Knows When its Lying. arXiv:2304.13734. · arXiv
  2. Bai, Y.; Kadavath, S.; Kundu, S.; Askell, A.; Kernion, J.; Jones, A.; Chen, A.; Goldie, A.; Mirhoseini, A.; McKinnon, C.; et al. 2022. Constitutional ai: Harmlessness from ai feed- back. arXiv preprint arXiv:2212.08073. · arXiv
  3. Flower, L.; and Hayes, J. R. 1981. A cognitive process the- ory of writing. College composition and communication, 32(4): 365-387.
  4. Ganguli, D.; Askell, A.; Schiefer, N.; Liao, T.; Lukošiūtė, K.; Chen, A.; Goldie, A.; Mirhoseini, A.; Olsson, C.; Hernandez, D.; et al. 2023. The capacity for moral self-correction in large language models. arXiv preprint arXiv:2302.07459. · arXiv
  5. PMLR.
  6. Guo, B.; Zhang, X.; Wang, Z.; Jiang, M.; Nie, J.; Ding, Y.; Yue, J.; and Wu, Y. 2023. How Close is ChatGPT to Hu- man Experts? Comparison Corpus, Evaluation, and Detec- tion. arXiv:2301.07597. · arXiv
  7. Holtzman, A.; Buys, J.; Du, L.; Forbes, M.; and Choi, Y. 2020. The Curious Case of Neural Text Degeneration. In International Conference on Learning Representations.
  8. Hu, E. J.; Shen, Y.; Wallis, P.; Allen-Zhu, Z.; Li, Y.; Wang, S.; Wang, L.; and Chen, W. 2021. Lora: Low-rank adaptation of large language models. In International Conference On Learning Representations.
  9. Huang, Y.; Feng, X.; Feng, X.; and Qin, B. 2023. The Fac- tual Inconsistency Problem in Abstractive Text Summariza- tion: A Survey. arXiv:2104.14839. · arXiv
  10. Kojima, T.; Gu, S. S.; Reid, M.; Matsuo, Y.; and Iwasawa, Y. 2022. Large language models are zero-shot reason- ers. Advances in neural information processing systems, 35: 22199-22213.
  11. Li, X. L.; and Liang, P. 2021. Prefix-tuning: Optimiz- ing continuous prompts for generation. arXiv preprint arXiv:2101.00190. · arXiv
  12. Lin, S.; Hilton, J.; and Evans, O. 2022. TruthfulQA: Measur- ing How Models Mimic Human Falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), 3214-3252. Dublin, Ireland: Association for Computational Linguistics.
  13. Madaan, A.; Tandon, N.; Gupta, P.; Hallinan, S.; Gao, L.; Wiegreffe, S.; Alon, U.; Dziri, N.; Prabhumoye, S.; Yang, Y.; Gupta, S.; Majumder, B. P.; Hermann, K.; Welleck, S.; Yazdanbakhsh, A.; and Clark, P. 2023. Self-Refine: Iterative Refinement with Self-Feedback. arXiv:2303.17651. · arXiv
  14. Manakul, P.; Liusie, A.; and Gales, M. J. F. 2023. SelfCheckGPT: Zero-Resource Black-Box Hallucina- tion Detection for Generative Large Language Models. arXiv:2303.08896. · arXiv
  15. Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C. L.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; Schulman, J.; Hilton, J.; Kelton, F.; Miller, L.; Simens, M.; Askell, A.; Welinder, P.; Christiano, P.; Leike, J.; and Lowe, R. 2022. Training language models to follow instruc- tions with human feedback. arXiv:2203.02155. · arXiv
  16. Paul, D.; Ismayilzada, M.; Peyrard, M.; Borges, B.; Bosselut, A.; West, R.; and Faltings, B. 2023. RE- FINER: Reasoning Feedback on Intermediate Representa- tions. arXiv:2304.01904. · arXiv
  17. Schick, T.; Dwivedi-Yu, J.; Jiang, Z.; Petroni, F.; Lewis, P.; Izacard, G.; You, Q.; Nalmpantis, C.; Grave, E.; and Riedel, S. 2023. PEER: A Collaborative Language Model. In In Proceedings of The 11th International Conference on Learn- ing Representations.
  18. Shinn, N.; Cassano, F.; Labash, B.; Gopinath, A.; Narasimhan, K.; and Yao, S. 2023. Reflexion: Lan- guage Agents with Verbal Reinforcement Learning. arXiv:2303.11366. · arXiv
  19. Stiennon, N.; Ouyang, L.; Wu, J.; Ziegler, D.; Lowe, R.; Voss, C.; Radford, A.; Amodei, D.; and Christiano, P. F. 2020. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33: 3008-3021.
  20. Suzgun, M.; Scales, N.; Schärli, N.; Gehrmann, S.; Tay, Y.; Chung, H. W.; Chowdhery, A.; Le, Q.; Chi, E.; Zhou, D.; and Wei, J. 2023. Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them. In Findings of the Association for Computational Linguistics: ACL 2023, 13003-13051. Toronto, Canada: Association for Computa- tional Linguistics.
  21. Wei, J.; Bosma, M.; Zhao, V. Y.; Guu, K.; Yu, A. W.; Lester, B.; Du, N.; Dai, A. M.; and Le, Q. V. 2022a. Finetuned Lan- guage Models Are Zero-Shot Learners. arXiv:2109.01652. · arXiv
  22. Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022b. Chain-of- thought prompting elicits reasoning in large language mod- els. Advances in Neural Information Processing Systems, 35: 24824-24837.
  23. Welleck, S.; Lu, X.; West, P.; Brahman, F.; Shen, T.; Khashabi, D.; and Choi, Y. 2022. Generating Sequences by Learning to Self-Correct. ArXiv, abs/2211.00053. · arXiv
  24. Weng, Y.; Zhu, M.; He, S.; Liu, K.; and Zhao, J. 2022. Large language models are reasoners with self-verification. arXiv preprint arXiv:2212.09561. · arXiv
  25. Yang, K.; Tian, Y.; Peng, N.; and Klein, D. 2022. Re3: Gen- erating Longer Stories With Recursive Reprompting and Re- vision. In Proceedings of the 2022 Conference on Empiri- cal Methods in Natural Language Processing, 4393-4479.
  26. Zhou, D.; Schärli, N.; Hou, L.; Wei, J.; Scales, N.; Wang, X.; Schuurmans, D.; Cui, C.; Bousquet, O.; Le, Q.; and Chi, E. 2023. Least-to-Most Prompting Enables Complex Rea- soning in Large Language Models. arXiv:2205.10625. · arXiv

Cited by

Sorted by citations

Authors
Han, H., Liang, J., Shi, J., He, Q., & Xiao, Y.
Affiliations
  • East China Normal University (CN)
  • Fudan University (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
Volume
38
Issue
16
Pages
18162-18170
ISSN
2159-5399 (ISSN-L)
DOI
10.48550/arxiv.2401.07301
arXiv
2401.07301
OpenAlex
W4393160208
Cited by
10
References
1
Citations by year
2025: 8 · 2024: 2
FWCI
2.89
Type
conference paper
Language
en
Open access
Licence
Locations
  • Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion, other-oa)
  • arXiv (Cornell University) (submittedVersion, other-oa)
  • AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Natural Science Foundation of China (2021SHZDZX0103, 62102095)
  • Fudan University (2021SHZDZX0103)
  • Science and Technology Commission of Shanghai Municipality (2021 SHZDZX0 103, No. 2021SHZDZX0103, 22511105902)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
self-correction, small language models, commonsense reasoning, hallucinations, fine-tuning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review