Self-Rewarding Language Models
Open accessAbstract
We posit that to achieve superhuman agents, future models require superhuman feedback in order to provide an adequate training signal. Current approaches commonly train reward models from human preferences, which may then be bottlenecked by human performance level, and secondly these separate frozen reward models cannot then learn to improve during LLM training. In this work, we study Self-Rewarding Language Models, where the language model itself is used via LLM-as-a-Judge prompting to provide its own rewards during training. We show that during Iterative DPO training that not only does instruction following ability improve, but also the ability to provide high-quality rewards to itself. Fine-tuning Llama 2 70B on three iterations of our approach yields a model that outperforms many existing systems on the AlpacaEval 2.0 leaderboard, including Claude 2, Gemini Pro, and GPT-4 0613. While there is much left still to explore, this work opens the door to the possibility of models that can continually improve in both axes.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Self-Rewarding Language Models, in which the model that generates responses also judges them so that both policy and reward signal improve across iterations, is the first widely-noted instance of a genuinely recursive training loop and the origin of its characteristic failure mode.
“Self-Rewarding Language Models [8], in which the model that generates responses also judges them, so that both the policy and the reward signal improve across iterations - the first widely-noted instance of a genuinely recursive training loop, and the origin of its characteristic failure mode.”
From Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops (2026) · cites this paper as [8]
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredRecent work co-adapts generators and rewards within gradient-based training of a single model or co-evolves evaluation metrics with agent skills under ground-truth guards.
“Recent work co-adapts generators and rewards within the gradientbased training of a single model [18-20] or co-evolves evaluation metrics with agent skills under ground-truth guards [21].”
From The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators (2026) · cites this paper as [18]
SPIN and Self-Rewarding Language Models use the same instance of the language models themselves as the reward model to progressively improve the generative and discriminative abilities of the same LLM for alignment.
“Most recently, SPIN and Self-Rewarding Language Models (Chen et al., 2024; Yuan et al., 2024) use the same instance of the language models themselves as the reward model to progressively improve the generative and discriminative abilities of the same LLM for alignment.”
From Absolute Zero: Reinforced Self-play Reasoning with Zero Data (2025) · cites this paper as Yuan et al., 2024
Information masses across prompts were consistent in Section 3.3.
“To ensure the information masses across prompts were consistent in Section 3.3, we followed an approach similar to the methodology in Yuan et al. (2024).”
From Explaining Length Bias in LLM-Based Preference Evaluations (2025) · cites this paper as Yuan et al. (2024)
Majority-agreed pseudo-label under self-consistency reduces stochastic or formatting errors.
“Among valid candidates, we adopt the majority-agreed function as a pseudo-label under self-consistency (Zuo et al., 2025; Huang et al., 2023; Yuan et al., 2024), since agreement across independent samples is less likely to arise from stochastic or formatting errors.”
From TT-SI: Self-Improving LLM Agents with Test-Time Training (2026) · cites this paper as Yuan et al., 2024
The Human evaluation section uses a majority vote of three human annotators to compare model responses, but does not describe majority agreement among sampled functions as a pseudo-label. Searches for self-consistency, pseudo-labels, and stochastic or formatting errors found no passage in our copy supporting the stated rationale. (LLM, conf=0.77)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 34
- Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. GPT-4 technical report. arXiv preprint arXiv:2303.08774, 2023. · arXiv
- Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022a. · arXiv
- Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitu- tional AI: Harmlessness from AI feedback. arXiv preprint arXiv:2212.08073, 2022b. · arXiv
- Yushi Bai, Jiahao Ying, Yixin Cao, Xin Lv, Yuze He, Xiaozhi Wang, Jifan Yu, Kaisheng Zeng, Yijia Xiao, Haozhe Lyu, Jiayin Zhang, Juanzi Li, and Lei Hou. Benchmarking foundation models with language-model-as-an-examiner. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2023. URL https://openreview.net/forum?id=IiRHQ7gvnq.
- Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. Piqa: Reasoning about physical commonsense in natural language. In Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020.
- Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, et al. AlpaGasus: Training a better alpaca with fewer data. In The Twelfth International Conference on Learning Representations, 2024a. URL https://openreview.net/forum?id=FdVXgSJhvz.
- Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. Self-play fine-tuning converts weak language models to strong language models. arXiv preprint arXiv:2401.01335, 2024b. · arXiv
- Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? Try ARC, the AI2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018. · arXiv
- Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. · arXiv
- Ronan Collobert and Jason Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th International Conference on Machine Learning, pages 160-167, 2008.
- Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Alpacafarm: A simulation framework for methods that learn from human feedback. arXiv preprint arXiv:2305.14387, 2023. · arXiv
- Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, et al. Reinforced self-training (rest) for language modeling. arXiv preprint arXiv:2308.08998, 2023. · arXiv
- Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Ja- cob Steinhardt. Measuring massive multitask language understanding. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https://openreview.net/forum?id=d7KBjmI3GmQ.
- Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, et al. Prometheus: Inducing fine-grained evaluation capability in language models. arXiv preprint arXiv:2310.08491, 2023. · arXiv
- Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi-Rui Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Richárd Nagyfi, et al. OpenAssistant conversations-democratizing large language model alignment. arXiv preprint arXiv:2304.07327, 2023. · arXiv
- Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Matthew Kelcey, Jacob Devlin, Kenton Lee, Kristina N. Toutanova, Llion Jones, Ming-Wei Chang, Andrew Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural questions: a benchmark for question answering research. Transactions of the Association of Computational Linguistics, 2019.
- Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Luke Zettlemoyer, Omer Levy, Jason Weston, and Mike Lewis. Self-alignment with instruction backtranslation. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview. net/forum?id=1oijHJBRsT.
- Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74-81, Barcelona, Spain, July 2004. Association for Computational Linguistics. URL https://aclanthology.org/W04-1013.
- Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. In EMNLP, 2018.
- Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744, 2022.
- Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. Automatically correcting large language models: Surveying the landscape of diverse self-correction strategies. arXiv preprint arXiv:2308.03188, 2023. · arXiv
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
- Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=HPuSIXJaa9.
- Swarnadeep Saha, Omer Levy, Asli Celikyilmaz, Mohit Bansal, Jason Weston, and Xian Li. Branch-solve-merge improves large language model evaluation and generation. arXiv preprint arXiv:2310.15123, 2023. · arXiv
- Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. Socialiqa: Commonsense reasoning about social interactions. CoRR, abs/1904.09728, 2019. URL http://arxiv.org/abs/1904.09728. · arXiv
- John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. · arXiv
- Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008-3021, 2020.
- Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023. · arXiv
- Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-SNE. Journal of machine learning research, 9(11), 2008.
- Wei Xiong, Hanze Dong, Chenlu Ye, Han Zhong, Nan Jiang, and Tong Zhang. Gibbs sampling from human feedback: A provable kl-constrained framework for rlhf. arXiv preprint arXiv:2312.11456, 2023. · arXiv
- Jing Xu, Andrew Lee, Sainbayar Sukhbaatar, and Jason Weston. Some things are more cringe than others: Preference optimization with the pairwise cringe loss. arXiv preprint arXiv:2312.16682, 2023. · arXiv
- Hongyi Yuan, Zheng Yuan, Chuanqi Tan, Wei Wang, Songfang Huang, and Fei Huang. RRHF: Rank responses to align language models with human feedback. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview. net/forum?id=EdIGMCHk4l.
- Yao Zhao, Rishabh Joshi, Tianqi Liu, Misha Khalman, Mohammad Saleh, and Peter J Liu. SLiC-HF: Sequence likelihood calibration with human feedback. arXiv preprint arXiv:2305.10425, 2023. · arXiv
- Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019. · arXiv
Cited by
Sorted by citations
Optimizing generative AI by backpropagating language model feedback
Yuksekgonul, M., Bianchi, F., Boen, J., Liu, S., Lu, P., Huang, Z., Guestrin, C., & Zou, J. (2025) · Nature · doi:10.1038/s41586-025-08661-4 · Cited by 85 · see citing works
A Survey on Symbolic Knowledge Distillation of Large Language Models
Acharya, K., Velasquez, A., & Song, H. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3428519 · Cited by 50 · see citing works
EduPlanner: LLM-Based Multiagent Systems for Customized and Intelligent Instructional Design
Zhang, X., Zhang, C., Sun, J., Xiao, J., Yang, Y., & Luo, Y. (2025) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2025.3561332 · Cited by 46 · see citing works
LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
Li, H., Dong, Q., Chen, J., Su, H., Zhou, Y., Ai, Q., Ye, Z., & Liu, Y. (2024) · arXiv · doi:10.48550/arxiv.2412.05579 · Cited by 40 · see citing works
Absolute Zero: Reinforced Self-play Reasoning with Zero Data
Zhao, A., Wu, Y., Yue, Y., Wu, T., Xu, Q., Yue, Y., Lin, M., Wang, S., Wu, Q., Zheng, Z., et al. (2025) · arXiv · Cited by 13 · see citing works
RewardBench: Evaluating Reward Models for Language Modeling
Lambert, N., Pyatkin, V., Morrison, J., Miranda, L., Lin, B. Y., Chandu, K., Dziri, N., Kumar, S., Zick, T., Choi, Y., et al. (2025) · Findings of the Association for Computational Linguistics: NAACL 2025 · doi:10.18653/v1/2025.findings-naacl.96 · Cited by 13 · see citing works
TextGrad: Automatic "Differentiation" via Text
Yuksekgonul, M., Bianchi, F., Boen, J., Liu, S., Huang, Z., Guestrin, C., & Zou, J. (2024) · arXiv · doi:10.48550/arxiv.2406.07496 · Cited by 12 · see citing works
Luo, Q., Lin, Q., Xu, L., Wu, S., Mao, R., Wang, C., Feng, H., Huang, B., & Du, Z. (2026) · Big Earth Data · doi:10.1080/20964471.2026.2615511 · Cited by 11 · see citing works
Shaowei, G., Hui, V. C. C., Štiglic, G., Constantino, R. E., Lee, Y. J., & Wong, A. K. C. (2025) · Journal of Medical Internet Research · doi:10.2196/65397 · Cited by 8 · see citing works
Training LLMs for Generating IEC 61131-3 Structured Text with Online Feedback
HAAG, A. L., Fuchs, B., Kacan, A., & Lohse, O. (2025) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1109/llm4code66737.2025.00013 · Cited by 8 · see citing works
A safety realignment framework via subspace-oriented model fusion for large language models
Yi, X., Zheng, S., Wang, L., Wang, X., & He, L. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112701 · Cited by 10 · see citing works
Use of Transfer Learning for Affordable In-Context Fake Review Generation
Ibañez-Lissen, L., González‐Manzano, L., Fuentes, J. M. D., & Goyanes, M. (2025) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3536927 · Cited by 6 · see citing works
D’Oosterlinck, K., Xu, W., Develder, C., Demeester, T., Singh, A., Potts, C., Kiela, D., & Mehri, S. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00748 · Cited by 12 · see citing works
Fang, J., Peng, Y., Zhang, X., Wang, Y., Yi, X., Zhang, G., Xu, Y., Wu, B., Liu, S., Li, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.07407 · Cited by 6 · see citing works
Learning to Judge: LLMs Designing and Applying Evaluation Rubrics
Siro, C., Aliannejadi, P., & Aliannejadi, M. (2026) · Findings of the Association for Computational Linguistics: EACL 2026 · doi:10.18653/v1/2026.findings-eacl.335 · Cited by 4 · see citing works
Explaining Length Bias in LLM-Based Preference Evaluations
Hu, Z., Song, L., Zhang, J., Xiao, Z., Wang, T., Chen, Z., Yuan, N. J., Lian, J., Ding, K., & Xiong, H. (2025) · Findings of the Association for Computational Linguistics: EMNLP 2025 · doi:10.18653/v1/2025.findings-emnlp.358 · Cited by 4 · see citing works
Knowledge Distillation and Transformer-Based Framework for Automatic Spine CT Report Generation
Batool, H., Mukhtar, A., Khawaja, S. G., Alghamdi, N. S., Khan, A. M., Qayyum, A., Adil, R., Khan, Z., Akram, M. U., Akbar, M. U., et al. (2025) · IEEE Access · doi:10.1109/access.2025.3546131 · Cited by 4 · see citing works
Gao, H.-A., Geng, J., Hua, W., Hu, M., Juan, X., Liu, H., Liu, S., Qiu, J., Qi, X., Wu, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2507.21046 · Cited by 3 · see citing works
Advanced evaluation of BIM-GenAI using OpenAI o1 and ethical considerations
Mirindi, D., Sinkhonde, D., Mirindi, F., & Bezabih, T. (2025) · ACM SIGCPR Conference on Computer Personnel Research (SIGCPR) · doi:10.1145/3716489.3728431 · Cited by 3 · see citing works
LLMs can Design Sustainable Concrete – a Systematic BenchmarkPreprint
Völker, C., Rug, T., Jablonka, K. M., & Kruschwitz, S. (2024) · Research Square · doi:10.21203/rs.3.rs-3913272/v1 · Cited by 4 · see citing works
Humanity Amplified: The Fusion of Deep Learning and Human Insight to Shape the Future of InnovationPreprint
Bauschard, S., & Quidwai, S. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4621210 · Cited by 2 · see citing works
TT-SI: Self-Improving LLM Agents with Test-Time Training
Acikgoz, E. C., Qian, C., Ji, H., Hakkani-Tür, D., & Tur, G. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.462 · Cited by 1 · see citing works
Harness^2: Recursive Agent Harnessing for an Open World
Xu, R., Chen, Y., CuiZhu, Z., Mishra, B. D., Ming, Y., Yu, H., Han, R., Lee, C.-Y., & Pfister, T. (2026) · arXiv · doi:10.48550/arxiv.2610.14224 · Cited by 1 · see citing works
“The object of such changes varies: model weights, as in self-rewarding language models (Yuan et al., 2024); improvement programs, as in STOP (Zelikman et al., 2024); and agent code, as in the Darwin Gödel Machine (Zhang et al., 2026b).”
The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators
Iacob, A., Jovanović, A., Shen, W. F., Burkhardt, D., Kurmanji, M., Tastan, N., Sani, L., Venanzi, N. A. E., Nie, J., Odonnat, A., et al. (2026) · arXiv · doi:10.48550/arxiv.2606.26294 · Cited by 1 · see citing works
“A task is eligible unless its dataset is exhausted under sampling; when sampling with replacement every task stays eligible ( D 𝑟, elig = D 𝑟 ), and a role is eligible when any task is.”
The superalignment of superhuman intelligence with large language models
Huang, M., Wang, Y., Cui, S., Ke, P., & Tang, J. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4348-6 · Cited by 1 · see citing works
Gómez Álvarez, S., Mozo Quesada, A., Navarro, T., Gálvez Rojas, S., & López Valverde, F. (2026) · Journal of Intelligent Information Systems · doi:10.1007/s10844-026-01086-z
LatentEvolve: Self-Evolving Test-Time Scaling in Latent Space
Zhang, G., Meng, F., Wan, G., Li, Z., Wang, K., Yin, Z., Bai, L., & Yan, S. (2025) · arXiv · doi:10.48550/arxiv.2509.24771
“…(training-free) : vanilla model and CoT (Wei et al., 2023); • Reinforcement Learning : (1) self-rewarding methods, including Self-Rewarding (Yuan et al., 2025) and Genius (Xu et al., 2025a), and (2) verifiable reward methods, including GRPO (DeepSeek-AI et al., 2025), Reinforce (Williams,…”
Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops
Chen, M., Wang, L., & Qu, B. (2026) · arXiv
“…bootstrapped rationales [7] through ReST^EM’s scaled self-training [84], SPIN’s self-play fine-tuning [85], and Self-Rewarding Language Models [8], in which the model that generates responses also judges them, so that both the policy and the reward signal improve across iterations — the first…”
Self-Improving LLM Agents at Test-Time
Acikgoz, E. C., Qian, C., Ji, H., Hakkani-Tür, D., & Tur, G. (2025) · arXiv
“Recent studies suggest that LLMs can self-improve their capabilities (Huang et al., 2023; Wang et al., 2023; Chen et al., 2024a; Pang et al., 2024; Yuan et al., 2024; Shafayat et al., 2025; Huang et al., 2025), i.e., they can refine their own output distribution using internal signals derived from…”
Li, Z., Li, Z., Shi, Y., Wang, R., Yang, J., Liu, Z., Wu, X., Li, A., Huang, C., Shang, E., et al. (2026) · arXiv · doi:10.48550/arxiv.2607.08964
Visual-ERM: Reward Modeling for Visual Equivalence
Liu, Z., Ding, S., Fang, X., Dai, X., Yang, P., Liang, J., Wang, J., Chen, K., Lin, D., & Zang, Y. (2026) · arXiv · doi:10.48550/arxiv.2603.13224
SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills
Zheng, B., Fatemi, M. Y., Jin, X., Wang, Z. Z., Gandhi, A., Song, Y., Gu, Y., Srinivasa, J., Liu, G., Neubig, G., et al. (2025) · arXiv · doi:10.48550/arxiv.2504.07079
Zhou, C., Zhang, X., Song, D., Chen, X., Gu, W., Ma, H., Tian, Y., Zhang, M., & Hu, L. (2025) · arXiv · doi:10.48550/arxiv.2502.09183
SJ-ORPO: Alignment of LLMs Using Multi-Task Learning and Self-Judgment
Parashar, J., & Bhandarkar, S. M. (2025) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai64502.2025.00040
Self-Improved Visual Preference Alignment by Iteratively Generating Holistic WinnerPreprint
Feng, Z., Chen, K., Zhang, J., Yang, J., & Song, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5352865
TTRL: Test-Time Reinforcement Learning
Zuo, Y., Zhang, K., Sheng, L., Qu, S., Cui, G., Zhu, X., Li, H., Zhang, Y., Long, X., Hua, E., et al. (2025) · arXiv · Cited by 7 · see citing works
“Previous studies have explored self-rewarding [32, 58] and self-play training [5] for unlabeled data.”
Self-Challenging Language Model Agents
Zhou, Y., Levine, S., Weston, J., Li, X., & Sukhbaatar, S. (2025) · arXiv · doi:10.48550/arxiv.2506.01716 · Cited by 1
Self-citation
- Authors
- Yuan, W., Pang, R. Y., Cho, K., Li, X., Sukhbaatar, S., Xu, J., & Weston, J.
- Affiliations
- New York University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2401.10020
- OpenAlex
- W4391047180
- Cited by
- 18
- References
- 0
- Citations by year
- 2026: 2 · 2025: 11 · 2024: 3 · 2023: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- reward modeling, LLM-as-a-judge, instruction following, human preference data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
