CoChat
Sign Up
Cited by 3Ethics and Social Impacts of AITopic ModelingHate Speech and Cyberbullying Detection

Smaller Large Language Models Can Do Moral Self-Correction

Open access
Liu, Guangliang, Xue, Zhiyu, Zhang, Xitong, Wang, Rongrong, Johnson, Kristen·2025·Proceedings of the 5th Workshop on Trustworthy NLP (TrustNLP 2025)
DOI 10.18653/v1/2025.trustnlp-main.5
Open this paper in CoChat — free accountOpen PDF

Abstract

Self-correction is one of the most amazing emerging capabilities of Large Language Models (LLMs), enabling LLMs to self-modify an inappropriate output given a natural language feedback which describes the problems of that output.Moral self-correction is a posthoc approach correcting unethical generations without requiring a gradient update, making it both computationally lightweight and capable of preserving the language modeling ability.Previous works have shown that LLMs can self-debias, and it has been reported that small models, i.e., those with less than 22B parameters, are not capable of moral self-correction.However, there is no direct proof as to why such smaller models fall short of moral selfcorrection, though previous research hypothesizes that larger models are skilled in following instructions and understanding abstract social norms.In this paper, we empirically validate this hypothesis in the context of social stereotyping, through meticulous prompting.Our experimental results indicate that (i) surprisingly, 3.8B LLMs with proper safety alignment fine-tuning can achieve very good moral selfcorrection performance, highlighting the significant effects of safety alignment; and (ii) small LLMs are indeed weaker than larger-scale models in terms of comprehending social norms and self-explanation through CoT, but all scales of LLMs show bad self-correction performance given unethical instructions.Content Warning: some examples in this paper are offensive or toxic.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2025.trustnlp-main.5

Cites

0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • SLMs can reliably handle high-frequency, structured decision tasks in online memory processing, enabling different memory stages to be assigned to models of appropriate scale.

    “SLMs can reliably handle high-frequency, structured decision tasks in online memory processing, enabling different memory stages to be assigned to models of appropriate scale (Liu et al., 2025).”

    From Lightweight LLM Agent Memory with Small Language Models (2026) · cites this paper as Liu et al., 2025

    The Introduction (page unavailable) reports that some smaller models can perform moral self-correction, while the Experimental Setting (page unavailable) compares model scales on social-bias benchmarks. Searches for high-frequency structured decisions, online memory processing, and assigning memory stages by model scale found no passage addressing those claims. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

  • SLMs can reliably handle high-frequency, structured decision tasks in online memory processing, enabling different memory stages to be assigned to models of appropriate scale.

    “SLMs can reliably handle high-frequency, structured decision tasks in online memory processing, enabling different memory stages to be assigned to models of appropriate scale (Liu et al., 2025).”

    From Lightweight LLM Agent Memory with Small Language Models (2026) · cites this paper as Liu et al., 2025

    The Introduction reports that some smaller models can perform moral self-correction, and the Experimental Setting compares models of different scales on social-bias tasks (pages not provided). A search of our copy found no passage about high-frequency decisions in online memory processing or assigning memory stages to models by scale. (LLM, conf=0.74)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 48

  1. Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harki- rat Behl, et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219. · arXiv
  2. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. · arXiv
  3. Emily M Bender, Timnit Gebru, Angelina McMillan- Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM confer- ence on fairness, accountability, and transparency, pages 610-623.
  4. Rajen Chatterjee, Matteo Negri, Raphael Rubino, and Marco Turchi. 2018. Findings of the WMT 2018 shared task on automatic post-editing. In Proceed- ings of the Third Conference on Machine Translation: Shared Task Papers, pages 710-725, Belgium, Brus- sels. Association for Computational Linguistics. · DOI
  5. Pinzhen Chen, Zhicheng Guo, Barry Haddow, and Ken- neth Heafield. 2023a. Iterative translation refine- ment with large language models. arXiv preprint arXiv:2306.03856. · arXiv
  6. Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. 2023b. Teaching large language mod- els to self-debug. arXiv preprint arXiv:2304.05128. · arXiv
  7. Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022. GPTQ: Accurate post-training compression for generative pretrained transformers. arXiv preprint arXiv:2210.17323. · arXiv
  8. Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas Liao, Kamilė Lukošiūtė, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hernandez, et al. 2023. The capacity for moral self- correction in large language models. arXiv preprint arXiv:2302.07459. · arXiv
  9. Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bha- gia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muen- nighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Nis- hant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, and Han- naneh Hajishirzi. 2024. Olmo: Accelerating the sci- ence of language models. Preprint.
  10. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xiny- ing Song, and Denny Zhou. 2023. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798. · arXiv
  11. Joel Jang, Seonghyeon Ye, and Minjoon Seo. 2023. Can large language models truly understand prompts? a case study with negated prompts. In Transfer Learning for Natural Language Processing Work- shop, pages 52-62. PMLR.
  12. Masahiro Kaneko, Danushka Bollegala, and Naoaki Okazaki. 2022. Debiasing isn't enough! -on the effectiveness of debiasing MLMs and their social biases in downstream tasks. In Proceedings of the 29th International Conference on Computational Lin- guistics, pages 1299-1310, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
  13. Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, et al. 2024. Training language models to self- correct via reinforcement learning. arXiv preprint arXiv:2409.12917. · arXiv
  14. Andrew Lee, Xiaoyan Bai, Itamar Pres, Martin Watten- berg, Jonathan K Kummerfeld, and Rada Mihalcea. 2024. A mechanistic understanding of alignment al- gorithms: A case study on dpo and toxicity. arXiv preprint arXiv:2401.01967. · arXiv
  15. Shiyang Li, Jun Yan, Hai Wang, Zheng Tang, Xi- ang Ren, Vijay Srinivasan, and Hongxia Jin. 2023. Instruction-following evaluation through verbalizer manipulation. arXiv preprint arXiv:2307.10558. · arXiv
  16. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let's verify step by step. arXiv preprint arXiv:2305.20050. · arXiv
  17. Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chan- dra Bhagavatula, and Yejin Choi. 2023. The unlock- ing spell on base llms: Rethinking alignment via in- context learning. arXiv preprint arXiv:2312.01552. · arXiv
  18. Guangliang Liu, Lei Jiang, Xitong Zhang, and Kris- ten Marie Johnson. 2025. Revealing the pragmatic dilemma for moral reasoning acquisition in language models. arXiv preprint arXiv:2502.16600. · arXiv
  19. Guangliang Liu, Haitao Mao, Bochuan Cao, Zhiyu Xue, Xitong Zhang, Rongrong Wang, Jiliang Tang, and Kristen Johnson. 2024a. On the intrinsic self- correction capability of llms: Uncertainty and latent concept. arXiv preprint arXiv:2406.02378. · arXiv
  20. Guangliang Liu, Haitao Mao, Jiliang Tang, and Kris- ten Marie Johnson. 2024b. Intrinsic self-correction for enhanced morality: An analysis of internal mech- anisms and the superficial hypothesis. arXiv preprint arXiv:2407.15286. · arXiv
  21. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv:1907.11692. · arXiv
  22. Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. 2023. The flan collection: Designing data and methods for effective instruction tuning. arXiv preprint arXiv:2301.13688. · arXiv
  23. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2023. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651. · arXiv
  24. Siru Ouyang, Zhuosheng Zhang, Bing Yan, Xuan Liu, Jiawei Han, and Lianhui Qin. 2023. Structured chem- istry reasoning with large language models. arXiv preprint arXiv:2311.09656. · arXiv
  25. Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. 2023. Automatically correcting large language models: Sur- veying the landscape of diverse self-correction strate- gies. arXiv preprint arXiv:2308.03188. · arXiv
  26. Baolin Peng, Chunyuan Li, Pengcheng He, Michel Gal- ley, and Jianfeng Gao. 2023. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277. · arXiv
  27. Valentina Pyatkin, Jena D Hwang, Vivek Srikumar, Xim- ing Lu, Liwei Jiang, Yejin Choi, and Chandra Bha- gavatula. 2023. Clarifydelphi: Reinforced clarifica- tion questions with defeasibility rewards for social and moral situations. In Proceedings of the 61st An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11253- 11271.
  28. Zimo Qi, Guangliang Liu, Kristen Marie Johnson, and Lu Cheng. 2024. Is moral self-correction an in- nate capability of large language models? a mech- anistic analysis to self-correction. arXiv preprint arXiv:2410.20513. · arXiv
  29. Yuxiao Qu, Tianjun Zhang, Naman Garg, and Aviral Kumar. 2024. Recursive introspection: Teaching language model agents how to self-improve. arXiv preprint arXiv:2407.18219. · arXiv
  30. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9.
  31. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290. · arXiv
  32. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. The Journal of Machine Learning Research, 21(1):5485-5551.
  33. Maarten Sap, Saadia Gabriel, Lianhui Qin, Dan Juraf- sky, Noah A Smith, and Yejin Choi. 2020. Social bias frames: Reasoning about social and power im- plications of language. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5477-5490.
  34. Timo Schick, Sahana Udupa, and Hinrich Schütze. 2021. Self-diagnosis and self-debiasing: A proposal for re- ducing corpus-based bias in nlp. Transactions of the Association for Computational Linguistics, 9:1408- 1424.
  35. Hugo Touvron, Louis Martin, Kevin Stone, Peter Al- bert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288. · arXiv
  36. Tianlu Wang, Ping Yu, Xiaoqing Ellen Tan, Sean O'Brien, Ramakanth Pasunuru, Jane Dwivedi-Yu, Olga Golovneva, Luke Zettlemoyer, Maryam Fazel- Zarandi, and Asli Celikyilmaz. 2023. Shepherd: A critic for language model generation. arXiv preprint arXiv:2308.04592. · arXiv
  37. Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, and Slav Petrov. 2020. Measuring and reducing gendered correlations in pre-trained models. arXiv preprint arXiv:2010.06032. · arXiv
  38. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits rea- soning in large language models. Advances in neural information processing systems, 35:24824-24837.
  39. Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, et al. 2023. Larger language models do in-context learning differently. arXiv preprint arXiv:2303.03846. · arXiv
  40. Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. 2021. Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359. · arXiv
  41. Sean Welleck, Ximing Lu, Peter West, Faeze Brah- man, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. 2022. Generating sequences by learning to self-correct. arXiv preprint arXiv:2211.00053. · arXiv
  42. Xuansheng Wu, Wenlin Yao, Jianshu Chen, Xiaoman Pan, Xiaoyang Wang, Ninghao Liu, and Dong Yu. 2023. From language modeling to instruction follow- ing: Understanding the behavior shift in llms after instruction tuning. arXiv preprint arXiv:2310.00492. · arXiv
  43. Zhongbin Xie and Thomas Lukasiewicz. 2023. An em- pirical analysis of parameter-efficient methods for de- biasing pre-trained language models. arXiv preprint arXiv:2306.04067. · arXiv
  44. Kevin Yang, Yuandong Tian, Nanyun Peng, and Dan Klein. 2022. Re3: Generating longer stories with recursive reprompting and revision. In Proceedings of EMNLP.
  45. Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. 2023. Evaluating large language models at evaluating instruction following. arXiv preprint arXiv:2310.07641. · arXiv
  46. Qingjie Zhang, Han Qiu, Di Wang, Haoting Qian, Yim- ing Li, Tianwei Zhang, and Minlie Huang. 2024. Understanding the dark side of llms' intrinsic self- correction. arXiv preprint arXiv:2412.14959. · arXiv
  47. Jieyu Zhao, Daniel Khashabi, Tushar Khot, Ashish Sab- harwal, and Kai-Wei Chang. 2021. Ethical-advice taker: Do language models understand natural lan- guage interventions? In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4158-4164.
  48. Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2023. Lima: Less is more for alignment. arXiv preprint arXiv:2305.11206. · arXiv

Cited by

Sorted by citations

Related

Authors
Liu, G., Xue, Z., Zhang, X., Wang, R., & Johnson, K.
Affiliations
  • Michigan State University (US)
  • University of California, Santa Barbara (US)
ORCID
Venue
Proceedings of the 5th Workshop on Trustworthy NLP (TrustNLP 2025)
Publisher
Association for Computational Linguistics
Published
2025
Pages
56-65
DOI
10.18653/v1/2025.trustnlp-main.5
OpenAlex
W4411120257
Cited by
1
References
0
Citations by year
2026: 1
FWCI
1.03
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Workshop on Trustworthy NLP (TrustNLP) (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
Keywords
large language models, small language models, social stereotyping, self-explanation, chain-of-thought prompting, model size effects

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review