DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Open accessAbstract
Mathematical reasoning poses a significant challenge for language models due to its complex and structured nature. In this paper, we introduce DeepSeekMath 7B, which continues pre-training DeepSeek-Coder-Base-v1.5 7B with 120B math-related tokens sourced from Common Crawl, together with natural language and code data. DeepSeekMath 7B has achieved an impressive score of 51.7% on the competition-level MATH benchmark without relying on external toolkits and voting techniques, approaching the performance level of Gemini-Ultra and GPT-4. Self-consistency over 64 samples from DeepSeekMath 7B achieves 60.9% on MATH. The mathematical reasoning capability of DeepSeekMath is attributed to two key factors: First, we harness the significant potential of publicly available web data through a meticulously engineered data selection pipeline. Second, we introduce Group Relative Policy Optimization (GRPO), a variant of Proximal Policy Optimization (PPO), that enhances mathematical reasoning abilities while concurrently optimizing the memory usage of PPO.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Recent studies primarily rely on rule-based verification, where agent trajectories are evaluated against predefined rules.
“Recent studies primarily rely on rule-based verification (Shao et al., 2024; Mroueh, 2025), where agent trajectories are evaluated against predefined rules (Wei et al., 2025).”
From AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation (2026) · cites this paper as Shao et al., 2024
The Shao et al. citation attaches to the claim that recent studies primarily rely on rule-based verification. Section 5.2.3 instead says that, in RL, the reward function is usually a neural reward model. The separate claim about evaluating agent trajectories is cited to Wei et al., not Shao et al.; no page number is available in our copy. (LLM, conf=0.83)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredGRPO enhances stability by optimizing policies based on the relative quality of sampled trajectories, removing the need for an explicit value function.
“Among recent advances, GRPO (Shao et al., 2024) enhances stability by optimizing policies based on the relative quality of sampled trajectories, removing the need for an explicit value function.”
From Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents (2026) · cites this paper as Shao et al., 2024
Section 4.1.1 describes GRPO as optimizing a policy using rewards relative to other sampled outputs and says it removes the need for an additional value-function approximation. It does not establish that GRPO enhances stability; its explicit statement about stabilizing training concerns the clipping parameter introduced in PPO. (LLM, conf=0.86)
Experimental — accuracy not yet measuredEarly methods for mid-training data curation prioritized high-value sources by up-sampling domain-specific corpora and curated reasoning data, particularly those with CoT trajectories, to strengthen relevant analytical and reasoning capabilities in specific domains.
“Early methods for mid-training data curation prioritized high-value sources by up-sampling domain-specific corpora and curated reasoning data, particularly those with CoT trajectories, to strengthen relevant analytical and reasoning capabilities in specific domains [30,176].”
From A Survey of Large Language Models (2026) · cites this paper as [30]
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredState-of-the-art approaches for controllable reasoning typically employ reinforcement learning frameworks, where a multi-objective reward function is designed to optimize correctness and length compliance jointly.
“State-of-the-art approaches [98, 2] typically employ reinforcement learning frameworks [116], where a multi-objective reward function is designed to optimize correctness and length compliance jointly.”
From Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality (2025) · cites this paper as [116]
The Introduction and §5.2.3 describe reinforcement learning for mathematical reasoning, supporting the citation's use as an example of an RL approach. The reward discussion in §5.2.3 and the correctness-based rule described under “Observation about Gradient Coefficient” do not describe a multi-objective reward that jointly optimizes correctness and length compliance; searches for length constraints and token budgets found no such passage. Page numbers were not available in our copy. (LLM, conf=0.88)
Experimental — accuracy not yet measuredOutcome-based RL like GRPO encourages diverse reasoning paths, including error and self-correction, given ground-truth feedback.
“Outcome-based RL like GRPO (Shao et al., 2024) addresses this by encouraging diverse reasoning paths, including error and self-correction, while given ground-truth feedback”
From Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models (2025) · cites this paper as Shao et al., 2024
Section 4.1.2, “Outcome Supervision RL with GRPO,” says GRPO samples multiple outputs and assigns rewards at the end of each output. “Observation about Gradient Coefficient” says its reward model is trained from answer-correctness judgments and that GRPO can reinforce or penalize responses. Neither passage establishes that GRPO encourages diverse reasoning paths or reasoning that includes errors and self-correction. (LLM, conf=0.85)
Experimental — accuracy not yet measured
RL algorithms like PPO and GRPO have effectively aligned LLM agents for complex tasks.
“Reinforcement learning (RL) algorithms like PPO (Schulman et al., 2017) and GRPO (Shao et al., 2024) have effectively aligned LLM agents for complex tasks.”
From Just-In-Time Reinforcement Learning: Continual Learning in LLM Agents Without Gradient Updates (2026) · cites this paper as Shao et al., 2024
Section 4.1 introduces GRPO as an effective RL algorithm, and the ‘Exploration and Analysis of Reinforcement Learning’ section reports improved performance for DeepSeekMath-Instruct, including on out-of-domain evaluations. That supports the effectiveness attributed to GRPO, but the paper establishes it for mathematical reasoning in LLMs, not alignment of LLM agents across complex tasks generally. The PPO citation in the sentence is to a different source. (LLM, conf=0.84)
Experimental — accuracy not yet measuredThe GRPO run halts itself at step 175 on ten consecutive zero-advantage batches.
“The run halts itself at step 175 on ten consecutive zero-advantage batches: once almost every sampled rollout in a group answers correctly, GRPO's group-relative advantage (and hence the gradient) goes to zero.”
From Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents (2026) · cites this paper as Shao et al., 2024
Section 4.1.2 says GRPO computes group-relative advantages by normalizing rewards within each group, supporting the general explanation about reduced advantage when rewards do not differ. But our copy does not report a run halting at step 175, a ten-batch stopping rule, or the claim that the entire gradient goes to zero; Section 4.1.1 includes a separate KL term in the objective. (LLM, conf=0.88)
Experimental — accuracy not yet measuredReinforcement learning with verifiable rewards has been shown to be an effective approach for improving mathematical reasoning and coding abilities in language models.
“Reinforcement learning with verifiable rewards has been shown to be an effective approach for improving mathematical reasoning and coding abilities in language models [15, 21, 8, 33, 10, 13]”
From Generalizing Verifiable Instruction Following (2025) · cites this paper as [21]
Section 5.2.2 reports that reinforcement learning significantly improved the instruction-tuned model, and the paper's mathematical benchmark results show higher scores for DeepSeekMath-RL than DeepSeekMath-Instruct. The 'Observation about Gradient Coefficient' section describes rewards based on answer correctness. However, the coding results in §5.1.1 concern code and math training, not an improvement in coding ability from reinforcement learning with verifiable rewards. (LLM, conf=0.88)
Experimental — accuracy not yet measuredGRPO eliminates the need for a separate value model, groups multiple rollouts for a single input prompt, and uses the mean of rewards as a baseline.
“One notable example is GRPO [167], which eliminates the need for a separate value model, which group multiple rollouts for a single input prompt and uses the mean of rewards as a baseline.”
From RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs (2026) · cites this paper as [167]
Capability internalization is achieved by learning from reasoning traces, through supervised fine-tuning or preference-guided optimization methods such as DPO and GRPO.
“Technically, capability internalization is achieved by learning from reasoning traces, through supervised fine-tuning (Wei et al., 2022; Zelikman et al., 2022; Schick et al., 2023; Mukherjee et al., 2023) or preference-guided optimization methods such as DPO (Rafailov et al., 2023; Tunstall et al., 2023; Yuan et al., 2024c; Grattafiori et al., 2024) and GRPO (Shao et al., 2024; DeepSeek-AI et al.,”
From Memory in the Age of AI Agents (2025) · cites this paper as Shao et al., 2024
Group Relative Policy Optimization (GRPO) avoids an explicit value function while maintaining PPO-style stability.
“This formulation avoids an explicit value function while maintaining PPO-style stability.”
From Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning (2025) · cites this paper as Shao et al., 2024
GRPO enables a streamlined approach to policy optimization, wherein the advantage function is derived by normalizing rewards within a cohort of responses generated for each prompt.
“GRPO enables a streamlined approach to policy optimization, wherein the advantage function is derived by normalizing rewards within a cohort of responses generated for each prompt.”
From Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning (2025) · cites this paper as Shao et al., 2024
Training with GRPO and PPO was unstable.
“We experimented with various on-policy methods such as Group Relative Policy Optimization (GRPO) [66] and Proximal Policy Optimization (PPO) [67], but found the training to be unstable.”
From Self-Adapting Language Models (2025) · cites this paper as [66]
Computing separate baselines for each of the six task-role configurations can be viewed as an interpolation between per-question baselines and a global baseline, allowing for more structured variance reduction tailored to each task setup.
“This can be viewed as an interpolation between per-question baselines, as in GRPO (Shao et al., 2024), and a global baseline, allowing for more structured variance reduction tailored to each task setup.”
From Absolute Zero: Reinforced Self-play Reasoning with Zero Data (2025) · cites this paper as Shao et al., 2024
Memory-R1 achieves strong performance with as few as 152 question-answer pairs.
“Both agents are fine-tuned using PPO (Schulman et al., 2017) or GRPO (Shao et al., 2024), achieving strong performance with as few as 152 question-answer pairs.”
From Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning (2026) · cites this paper as Shao et al., 2024
DeepSeek-R1 demonstrates the feasibility of pure RL with Group Relative Policy Optimisation when ground-truth verification is possible.
“Notably, DeepSeek-R1 (Guo et al., 2025) further demonstrates the feasibility of pure RL with Group Relative Policy Optimisation (Shao et al., 2024) when ground-truth verification is possible.”
From A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems (2025) · cites this paper as Shao et al., 2024
iCLP achieves performance competitive with GRPO, which relies on reinforcement learning.
“achieving performance competitive with GRPO (Shao et al., 2024), which relies on reinforcement learning.”
From iCLP: Large Language Model Reasoning with Implicit Cognition Latent Planning (2025) · cites this paper as Shao et al., 2024
In normal mode and cross mode, iCLP consistently outperforms all baselines and closely matches the performance of the GRPO algorithm.
“In the normal mode and the more challenging cross mode , iCLP consistently outperforms all baselines and closely matches the performance of the GRPO algorithm (Shao et al., 2024).”
From iCLP: Large Language Model Reasoning with Implicit Cognition Latent Planning (2025) · cites this paper as Shao et al., 2024
Qwen3 non-thinking mode still exhibits the blind spot despite continued fine-tuning after GRPO to unify thinking and non-thinking modes.
“non-thinking mode still exhibits the blind spot.”
From Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models (2025) · cites this paper as Shao et al., 2024
Section 4.1.4 describes iterative GRPO training for DeepSeekMath, not Qwen3. Searches for Qwen3, non-thinking mode, and a persistent blind spot found no passage addressing the attributed claim in our copy. (LLM, conf=0.77)
Experimental — accuracy not yet measuredOn the real, held-out LongMemEval oracle set, GRPO fine-tuning Qwen2.5-3B on Supersede raises accuracy from 9.0% to 16.7% (13/78 vs 7/78 correct), a +7.7-point (+86% relative) gain.
“held-out oracle accuracy rises from 9.0% to 16.7% (13/78 vs. 7/78 correct), a + 7.7-point ( + 86% relative) gain that nearly doubles the untrained baseline”
From Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents (2026) · cites this paper as Shao et al., 2024
Section 4.1.1 describes GRPO, but the paper’s reported evaluations concern DeepSeekMath on mathematical benchmarks, not Qwen2.5-3B on LongMemEval. Searches for Supersede, the held-out oracle set, and the 7/78 and 13/78 results found no passage supporting the claimed accuracy gain. (LLM, conf=0.76)
Experimental — accuracy not yet measuredOn the procedural training distribution, GRPO reward climbs from 0.66 to 0.97.
“On the procedural training distribution, GRPO reward climbs from 0.66 to 0.97 (Figure 4a).”
From Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents (2026) · cites this paper as Shao et al., 2024
Section 4.1.1 describes how GRPO uses rewards, and the ‘Observation about Gradient Coefficient’ section discusses its performance relative to online RFT. Neither reports a procedural training distribution or a reward increase from 0.66 to 0.97; searches for those numbers and Figure 4a found no supporting passage in our copy. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 42
- J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021. · arXiv
- Z. Azerbayev, H. Schoelkopf, K. Paster, M. D. Santos, S. McAleer, A. Q. Jiang, J. Deng, S. Bider- man, and S. Welleck. Llemma: An open language model for mathematics. arXiv preprint arXiv:2310.10631, 2023. · arXiv
- J. Bai, S. Bai, Y. Chu, Z. Cui, K. Dang, X. Deng, Y. Fan, W. Ge, Y. Han, F. Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023. · arXiv
- C. Burns, P. Izmailov, J. H. Kirchner, B. Baker, L. Gao, L. Aschenbrenner, Y. Chen, A. Ecoffet, M. Joglekar, J. Leike, et al. Weak-to-strong generalization: Eliciting strong capabilities with weak supervision. arXiv preprint arXiv:2312.09390, 2023. · arXiv
- M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba. Evaluating large language models trained on code. CoRR, abs/2107.03374, 2021. URL https://arxiv.org/abs/2107.03374. · arXiv
- K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. · arXiv
- Z. Du, Y. Qian, X. Liu, M. Ding, J. Qiu, Z. Yang, and J. Tang. Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 320-335, 2022.
- L. Gao, A. Madaan, S. Zhou, U. Alon, P. Liu, Y. Yang, J. Callan, and G. Neubig. PAL: program- aided language models. In A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett, editors, International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 10764-10799. PMLR, 2023. URL https://proceedings.mlr.press/v202/gao23f. html.
- D. Guo, Q. Zhu, D. Yang, Z. Xie, K. Dong, W. Zhang, G. Chen, X. Bi, Y. Wu, Y. K. Li, F. Luo, Y. Xiong, and W. Liang. Deepseek-coder: When the large language model meets programming -the rise of code intelligence, 2024. · DOI · Paper page
- D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020. · arXiv
- D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. Mea- suring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. · arXiv
- A. Q. Jiang, S. Welleck, J. P. Zhou, W. Li, J. Liu, M. Jamnik, T. Lacroix, Y. Wu, and G. Lample. Draft, sketch, and prove: Guiding formal theorem provers with informal proofs. arXiv preprint arXiv:2210.12283, 2022. · arXiv
- A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023. · arXiv
- A. Joulin, E. Grave, P. Bojanowski, M. Douze, H. Jégou, and T. Mikolov. Fasttext. zip: Compress- ing text classification models. arXiv preprint arXiv:1612.03651, 2016. · arXiv
- W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
- Y. Leviathan, M. Kalman, and Y. Matias. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pages 19274-19286.
- PMLR, 2023.
- A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. Ramasesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, et al. Solving quantitative reasoning problems with language models. Advances in Neural Information Processing Systems, 35:3843-3857, 2022a.
- A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. V. Ramasesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, Y. Wu, B. Neyshabur, G. Gur-Ari, and V. Misra. Solving quantitative reasoning problems with language models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 -December 9, 2022, 2022b. URL http://papers.nips. cc/paper_files/paper/2022/hash/18abbeef8cfe9203fdf9053c9c4fe191-Abstr act-Conference.html.
- H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe. Let's verify step by step. arXiv preprint arXiv:2305.20050, 2023. · arXiv
- I. Loshchilov and F. Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. · arXiv
- H. Luo, Q. Sun, C. Xu, P. Zhao, J. Lou, C. Tao, X. Geng, Q. Lin, S. Chen, and D. Zhang. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint arXiv:2308.09583, 2023. · arXiv
- X. Nguyen, W. Zhang, X. Li, M. M. Aljunied, Q. Tan, L. Cheng, G. Chen, Y. Deng, S. Yang, C. Liu, H. Zhang, and L. Bing. Seallms -large language models for southeast asia. CoRR, abs/2312.00738, 2023. doi: 10.48550/ARXIV.2312.00738. URL https://doi.org/10.485 50/arXiv.2312.00738. · DOI
- OpenAI. GPT4 technical report. arXiv preprint arXiv:2303.08774, 2023. · arXiv
- L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744, 2022.
- L. C. Paulson. Three years of experience with sledgehammer, a practical link between auto- matic and interactive theorem provers. In R. A. Schmidt, S. Schulz, and B. Konev, editors, Proceedings of the 2nd Workshop on Practical Aspects of Automated Reasoning, PAAR-2010, Edinburgh, Scotland, UK, July 14, 2010, volume 9 of EPiC Series in Computing, pages 1-10. EasyChair, 2010. doi: 10.29007/TNFD. URL https://doi.org/10.29007/tnfd. · DOI
- S. Polu and I. Sutskever. Generative language modeling for automated theorem proving. CoRR, abs/2009.03393, 2020. URL https://arxiv.org/abs/2009.03393. · arXiv
- J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015. · arXiv
- J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. · arXiv
- F. Shi, M. Suzgun, M. Freitag, X. Wang, S. Srivats, S. Vosoughi, H. W. Chung, Y. Tay, S. Ruder, D. Zhou, D. Das, and J. Wei. Language models are multilingual chain-of-thought reasoners. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https://openreview.net/pdf?id= fR3wGCk-IXp.
- F. Song, B. Yu, M. Li, H. Yu, F. Huang, Y. Li, and H. Wang. Preference ranking optimization for human alignment. arXiv preprint arXiv:2306.17492, 2023. · arXiv
- M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261, 2022. · arXiv
- T. H. Trinh, Y. Wu, Q. V. Le, H. He, and T. Luong. Solving olympiad geometry without human demonstrations. Nature, 625(7995):476-482, 2024.
- P. Wang, L. Li, L. Chen, F. Song, B. Lin, Y. Cao, T. Liu, and Z. Sui. Making large language models better reasoners with alignment. arXiv preprint arXiv:2309.02144, 2023a. · arXiv
- P. Wang, L. Li, Z. Shao, R. Xu, D. Dai, Y. Li, D. Chen, Y. Wu, and Z. Sui. Math-shepherd: Verify and reinforce llms step-by-step without human annotations. CoRR, abs/2312.08935, 2023b. · arXiv
- J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS, 2022. URL http://papers.nips.cc/paper_files/paper/2022/hash/9d5609613524ecf 4f15af0f7b31abca4-Abstract-Conference.html.
- H. Xia, T. Ge, P. Wang, S.-Q. Chen, F. Wei, and Z. Sui. Speculative decoding: Exploiting speculative execution for accelerating seq2seq generation. In H. Bouamor, J. Pino, and K. Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 3909- 3925, Singapore, Dec. 2023. Association for Computational Linguistics. doi: 10.18653/v1/20 · DOI
- H. Xia, Z. Yang, Q. Dong, P. Wang, Y. Li, T. Ge, T. Liu, W. Li, and Z. Sui. Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding. arXiv preprint arXiv:2401.07851, 2024. · arXiv
- S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y. Cao, and K. Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601, 2023. · arXiv
- Z. Yuan, H. Yuan, C. Li, G. Dong, C. Tan, and C. Zhou. Scaling relationship on learning mathematical reasoning with large language models. arXiv preprint arXiv:2308.01825, 2023a. · arXiv
- Z. Yuan, H. Yuan, C. Tan, W. Wang, S. Huang, and F. Huang. Rrhf: Rank responses to align language models with human feedback without tears. arXiv preprint arXiv:2304.05302, 2023b. · arXiv
- K. Zheng, J. M. Han, and S. Polu. Minif2f: a cross-system benchmark for formal olympiad-level mathematics. arXiv preprint arXiv:2109.00110, 2021. · arXiv
Cited by
Sorted by citations
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093 · see citing works
Self-citationDeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al. (2025) · Nature · doi:10.1038/s41586-025-09422-z · Cited by 1,701 · see citing works
Self-citationGenerative artificial intelligence in medicine
Teo, Z. L., Thirunavukarasu, A. J., Elangovan, K., Cheng, H., Moova, P., Soetikno, B., Nielsen, C., Pollreisz, A., Ting, D. S. J., Morris, R. J. T., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03983-2 · Cited by 119 · see citing works
Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.09388 · Cited by 111 · see citing works
Self-citationExploring the role of large language models in the scientific method: from hypothesis to discovery
Zhang, Y., Khan, S. A., Mahmud, A., Yang, H., Lavin, A., Levin, M. G., Frey, J. G., Dunnmon, J., Evans, J., Bundy, A., et al. (2025) · npj Artificial Intelligence · doi:10.1038/s44387-025-00019-5 · Cited by 109 · see citing works
RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs
Chaudhari, S., Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K., Deshpande, A., & Castro da Silva, B. (2026) · ACM Computing Surveys · doi:10.1145/3743127 · Cited by 92 · see citing works
Toward large reasoning models: A survey of reinforced reasoning with large language models
Xu, F., Hao, Q., Shao, C., Zong, Z., Li, Y., Wang, J., Zhang, Y., Wang, J., Lan, X., Gong, J., et al. (2025) · Patterns · doi:10.1016/j.patter.2025.101370 · Cited by 186 · see citing works
Self-citationRahman, A., Mahir, S. H., Tashrif, M. T. A., Karim, M. A., Aishi, A. A., Kundu, D., Debnath, T., Moududi, M. A. A., Eidmum, M. Z. A., Miah, A. S. M., et al. (2025) · Systems and Soft Computing · doi:10.1016/j.sasc.2025.200396 · Cited by 81 · see citing works
Shojaee, P., Mirzadeh, I., Alizadeh vahid, K., Horton, M., Bengio, S., & Farajtabar, M. (2025) · Advances in Neural Information Processing Systems 38 · doi:10.52202/085713-3603 · Cited by 47 · see citing works
Scaling Large Language Models for Next-Generation Single-Cell AnalysisPreprint
Rizvi, S. A., Levine, D., Patel, A. M., Zhang, S., Wang, E., Perry, C. J., Vrkic, I., Constante, N. M., Fu, Z., He, S., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.04.14.648850 · Cited by 49 · see citing works
Aydın, Ö., Karaarslan, E., Erenay, F. S., & Bačanin, N. (2026) · Turkish Journal of Engineering · doi:10.31127/tuje.1838765 · Cited by 55 · see citing works
Reasoning beyond limits: Advances and open problems for LLMs
Ferrag, M. A., Tihanyi, N., & Debbah, M. A. (2025) · ICT Express · doi:10.1016/j.icte.2025.09.003 · Cited by 33 · see citing works
Critique of impure reason: Unveiling the reasoning behaviour of medical large language models
Sim, S. Z. Y., & Chen, T. (2025) · eLife · doi:10.7554/elife.106187 · Cited by 24 · see citing works
DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
DeepSeek-AI, Liu, A., Mei, A., Lin, B., Xue, B., Wang, B., Xu, B., Wu, B., Zhang, B., Lin, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2512.02556 · Cited by 19 · see citing works
“2, we still adopt Group Relative Policy Optimization (GRPO) (DeepSeek-AI, 2025; Shao et al., 2024) as the RL training algorithm.”
Self-citationTransforming agency: On the mode of existence of large language models
Barandiaran, X. E., & Almendros, L. S. (2025) · Phenomenology and the Cognitive Sciences · doi:10.1007/s11097-025-10094-3 · Cited by 24 · see citing works
Training Large Language Models to Reason in a Continuous Latent Space
Hao, S., Sukhbaatar, S., Su, D., Li, X., Hu, Z., Weston, J., & Tian, Y. (2024) · arXiv · doi:10.48550/arxiv.2412.06769 · Cited by 18 · see citing works
“…prompting LLMs (Wei et al., 2022; Khot et al., 2022; Zhou et al., 2022), or training LLMs to generate reasoning chains, either with supervised finetuning (Yue et al., 2023; Yu et al., 2023) or reinforcement learning (Wang et al., 2024; Havrilla et al., 2024; Shao et al., 2024; Yu et al., 2024a).”
SciCode: A Research Coding Benchmark Curated by Scientists
Tian, M., Gao, L., Zhang, S. D., Chen, X., Fan, C., Guo, X., Haas, R., Ji, P., Krongchon, K., Li, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.13168 · Cited by 15 · see citing works
Self-citationAbsolute Zero: Reinforced Self-play Reasoning with Zero Data
Zhao, A., Wu, Y., Yue, Y., Wu, T., Xu, Q., Yue, Y., Lin, M., Wang, S., Wu, Q., Zheng, Z., et al. (2025) · arXiv · Cited by 13 · see citing works
Self-citationZheng, X., Yang, J., Zhou, L., Qin, L., Lu, C., Alonso‐Vante, N., Huang, Z., Zhang, P., Zhou, Y., Luo, Z.-H., et al. (2025) · Chemical Engineering Science · doi:10.1016/j.ces.2025.121740 · Cited by 16 · see citing works
Wang, S., Yu, L., Gao, C., Zheng, C., Liu, S., Lu, R., Dang, K., Chen, X., Yang, J., Zhang, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.01939 · Cited by 14
“The estimated advantage ˆ A it is then computed as: , where In addition to this modified advantage estimation, GRPO adds a KL penalty term to the clipped objective in Equation (2).”
InfluentialLarge Language Models for Combinatorial Optimization: A Systematic Review
Ros, F. D., Soprano, M., Gaspero, L. D., & Roitero, K. (2026) · ACM Computing Surveys · doi:10.1145/3801961 · Cited by 24 · see citing works
Wang, P., Holmes, J., Liu, Z., Chen, D., Liu, T., Shen, J., & Liu, W. (2025) · Frontiers in Oncology · doi:10.3389/fonc.2025.1557064 · Cited by 11 · see citing works
Self-citationFunctional alignment of protein language models via reinforcement learning
Blalock, N., Seshadri, S., Nakamura, K., Babbar, A., Fahlberg, S. A., Kulkarni, A., & Romero, P. A. (2026) · Nature Communications · doi:10.1038/s41467-026-77557-2 · Cited by 14 · see citing works
The Hallucination Tax of Reinforcement Finetuning
Song, L., Shi, T., & Zhao, J. (2025) · Findings of the Association for Computational Linguistics: EMNLP 2025 · doi:10.18653/v1/2025.findings-emnlp.112 · Cited by 10 · see citing works
Luo, X., Huang, Y., Cui, J., & Zheng, K. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130795 · Cited by 12 · see citing works
General-Purpose Models for the Chemical Sciences: LLMs and Beyond
Alampara, N., Aneesh, A., Ríos-García, M., Mirza, A., Schilling-Wilhelmi, M., Aghajani, A. A., Sun, M., Prastalo, G., & Jablonka, K. M. (2026) · Chemical Reviews · doi:10.1021/acs.chemrev.5c00583 · Cited by 12 · see citing works
A Survey of Post-Training Scaling in Large Language Models
Lai, H., Liu, X., Gao, J., Cheng, J., Qi, Z., Xu, Y., Yao, S., Zhang, D., Du, J., Hou, Z., et al. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.140 · Cited by 8 · see citing works
Yan, S., Yang, X., Huang, Z., Nie, E., Ding, Z., Li, Z., Ma, X., Bi, J., Kersting, K., Pan, J. Z., et al. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.583 · Cited by 10 · see citing works
GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
Agrawal, L. A., Tan, S., Soylu, D., Ziems, N., Khare, R., Opsahl-Ong, K., Singhvi, A., Shandilya, H., Ryan, M. J., Jiang, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2507.19457 · Cited by 7 · see citing works
Jin, S., Fu, Z., Song, G., Pei, J., Gong, H., Ma, L., & Zhang, D. (2025) · Corrosion Science · doi:10.1016/j.corsci.2025.113217 · Cited by 7 · see citing works
Zhu, Y., Shu, X., Liu, X., Li, Y., Yi, B., & Ma, D. (2025) · Anesthesiology and Perioperative Science · doi:10.1007/s44254-025-00156-9 · Cited by 7 · see citing works
Self-citationBenchmarking Benchmark Leakage in Large Language Models
Xu, R., Wang, Z., Fan, R.-Z., & Liu, P. (2024) · arXiv · doi:10.48550/arxiv.2404.18824 · Cited by 6 · see citing works
Self-citationFang, J., Peng, Y., Zhang, X., Wang, Y., Yi, X., Zhang, G., Xu, Y., Wu, B., Liu, S., Li, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.07407 · Cited by 6 · see citing works
Ce‐LLMs: Status and trends of education‐specific large language models developed in China
Xie, T., Zhou, Y., & Yu, J. (2025) · Future in Educational Research · doi:10.1002/fer3.70008 · Cited by 7 · see citing works
DeepSeek in Education: Exploring the Transformative Potential of AI‐Driven Educational Intelligence
Liao, J., Sun, F., Liu, Y., & Hu, Y. (2025) · Future in Educational Research · doi:10.1002/fer3.70022 · Cited by 6 · see citing works
Agentic Lab: An Agentic-physical AI system for cell and organoid experimentation and manufacturingPreprint
Wang, W., Swain, S., Lee, J., Lin, Z., Canales, B., Aljović, A., Liu, Y., Li, Q., Marin‐Llobet, A., Liu, M., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.11.11.686354 · Cited by 8 · see citing works
Quantifying artificial intelligence through algorithmic generalization
Ito, T., Campbell, M. S., Lior Horesh, C., Klinger, T., & Ram, P. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01092-w · Cited by 6 · see citing works
DeepSeek: implications for data science and management in the AI era
Xu, Z., Liu, S., Huang, W., Shi, J., Xu, F., Tang, X., & Lu, H. (2025) · Data Science and Management · doi:10.1016/j.dsm.2025.09.001 · Cited by 5 · see citing works
WisWheat: A Three-Tiered Vision-Language Dataset for Wheat Management
Yuan, B., Song, S., Fernández, J. A., Luo, Y., Baktashmotlagh, M., & Huang, Z. H. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3758256 · Cited by 5 · see citing works
HPC-R1: Characterizing R1-like Large Reasoning Models on HPC
Weingram, A., Zhang, D., Chen, Z., Qi, H., & Lu, X. (2025) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3712285.3759827 · Cited by 6 · see citing works
Awan, S. A., Khan, M. A., Khan, A. A., Sathio, A. A., Alsayaydeh, J. A. J., Bacarra, R., Herawan, S. G., & Aziz, R. (2025) · Journal Of Big Data · doi:10.1186/s40537-025-01330-3 · Cited by 5 · see citing works
Synthetic4Health: generating annotated synthetic clinical letters
Ren, L., Belkadi, S., Han, L., Del-Pinto, W., & Nenadić, G. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1497130 · Cited by 8 · see citing works
LIMO: Less is More for Reasoning
Ye, Y., Huang, Z., Xiao, Y., Chern, E., Xia, S., & Liu, P. (2025) · arXiv · doi:10.48550/arxiv.2502.03387 · Cited by 4 · see citing works
“Two key developments have created conditions for reimagining reasoning in LLMs: (1) Knowledge foundation revolution : Modern foundation models now incorporate unprecedented amounts of mathematical content during pre-training (Qwen et al., 2025; Yang et al., 2024; Wang et al., 2024).”
de Almeida, V. A., de Camargo, V., Gómez-Bravo, R., van Boven, K., van der Haring, E., Finger, M., & López, L. F. (2026) · JAMIA Open · doi:10.1093/jamiaopen/ooag017 · Cited by 4 · see citing works
Exploring Text-to-Motion Generation with Human Preference
Sheng, J., Lin, M. G., Zhao, A., Pruvost, K., Wen, Y., Li, Y., Huang, G. M., & Liu, Y. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00195 · Cited by 4 · see citing works
Self-citationE2ETune: End-to-End Knob Tuning via Fine-Tuned Generative Language Model
Huang, X., Li, H., Zhang, J., Zhao, X., Yao, Z., Li, Y., Zhang, T., Chen, J., Chen, H., & Li, C. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3773731.3773732 · Cited by 4 · see citing works
Self-citationEnhancing math reasoning ability of large language models via computation logic graphs
Zhao, D., Han, D., Wu, J. X., He, Z., Ning, B., Yuan, Y., Li, Y., Wang, C., & Song, S. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113905 · Cited by 4 · see citing works
Self-citationGeneralizing Verifiable Instruction Following
Pyatkin, V., Malik, S., Graf, V., Ivison, H., Huang, S., Dasigi, P., Lambert, N., & Hajishirzi, H. (2025) · arXiv · doi:10.48550/arxiv.2507.02833 · Cited by 4 · see citing works
Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
Yue, Y., Chen, Z., Lu, R., Zhao, A., Wang, Z., Yue, Y., Song, S., & Huang, G. (2025) · arXiv · doi:10.48550/arxiv.2504.13837 · Cited by 3 · see citing works
“Recently, Reinforcement Learning with Verifiable Rewards (RLVR) has gained significant traction as a method to improve the reasoning capabilities of LLMs in domains such as mathematics and programming (Lambert et al., 2024; Shao et al., 2024).”
InfluentialZweiger, A., Pari, J., Guo, H., Akyürek, E., Kim, Y., & Agrawal, P. (2025) · arXiv · Cited by 3 · see citing works
“We experimented with various on-policy methods such as Group Relative Policy Optimization (GRPO) [66] and Proximal Policy Optimization (PPO) [67], but found the training to be unstable.”
Large Language Model Reasoning Failures
Song, P., Han, P., & Goodman, N. (2026) · arXiv · doi:10.48550/arxiv.2602.06176 · Cited by 3 · see citing works
Gao, H.-A., Geng, J., Hua, W., Hu, M., Juan, X., Liu, H., Liu, S., Qiu, J., Qi, X., Wu, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2507.21046 · Cited by 3 · see citing works
Self-citationSeed-Coder: Let the Code Model Curate Data for Itself
Seed, B., Zhang, Y., Su, J., Sun, Y., Xi, C., Xiao, X., Zheng, S., Zhang, A., Liu, K., Zan, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.03524 · Cited by 3 · see citing works
“During the RL training process, similar to DeepCoder [Luo et al., 2025], we used the open-source verl framework [Sheng et al., 2025] for GRPO training [Shao et al., 2024], and adopted optimization techniques similar to DAPO [Yu et al., 2025, ByteDance Seed, 2025].”
Self-citationStop Overthinking: A Survey on Efficient Reasoning for Large Language Models
Sui, Y., Chuang, Y.-N., Wang, G., Zhang, J., Zhang, T., Yuan, J., Liu, H., Wen, A., Zhong, S., Zou, N., et al. (2025) · arXiv · Cited by 2 · see citing works
Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory
Long, L., He, Y., Ye, W., Pan, Y., Lin, Y., Li, H., Zhao, J., & Li, W. (2025) · arXiv · doi:10.48550/arxiv.2508.09736 · Cited by 2 · see citing works
“We also use Group Relative Policy Optimization (GRPO)[37] to optimize the policy model in the ablation study.”
Yan, S., Yang, X., Huang, Z., Nie, E., Ding, Z., Li, Z., Ma, X., Bi, J., Kersting, K., Pan, J. Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.19828 · Cited by 2 · see citing works
“Both agents are fine-tuned using PPO (Schul-man et al., 2017) or GRPO (Shao et al., 2024), achieving strong performance with as few as 152 question–answer pairs.”
Demystifying Multi-Agent Debate: The Role of Confidence and Diversity
Zhu, X., Zhang, C., Chi, Y., Stafford, T., Collier, N., & Vlachos, A. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.1694 · Cited by 2 · see citing works
When DeepSeek-R1 meets financial applications: benchmarking, opportunities, and limitations
Liu, S., Chen, L., Yan, J., Jiang, Y., Wang, X., Li, X., & Yang, Q. (2025) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2500227 · Cited by 2 · see citing works
Leveraging label semantics and meta-label refinement for multi-label question classification
Dong, S., Niu, X., Zhong, R., Wang, Z., & Zuo, M. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113412 · Cited by 2 · see citing works
VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations for Synthetic VideosPreprint
Li, Z., Wu, X., Qin, Y., Shi, G., Du, H., Manocha, D., Zhou, T., & Boyd-Graber, J. L. (2025) · Qeios · doi:10.32388/bxc6x1 · Cited by 2 · see citing works
Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning
NVIDIA, :, Azzolini, A., Bai, J., Brandon, H., Cao, J., Chattopadhyay, P., Chen, H., Chu, J., Cui, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.15558 · Cited by 1 · see citing works
Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination
Wu, M., Zhang, Z., Dong, Q., Xi, Z., Zhao, J., Jin, S., Fan, X., Zhou, Y., Lv, H., Zhang, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2507.10532 · Cited by 1 · see citing works
“However, these methods fails to transfer to Llama or OLMo (OLMo et al. 2025), suggesting model-specific idiosyncrasies.”
Self-citationOh, G., Kim, S., Park, S., & Kim, B.-H. (2026) · Journal of Medical Internet Research · doi:10.2196/90693 · Cited by 1 · see citing works
“Med-R1 [32] uses group relative policy optimization [23] to improve generalizability across various medical imaging modalities, achieving a 29.94% accuracy improvement.”
InfluentialMiMo: Unlocking the Reasoning Potential of Language Model - From Pretraining to Posttraining
Xiaomi, L.-C., :, Xia, B., Shen, B., Cici, Zhu, D., Zhang, D., Wang, G., Zhang, H., Liu, H., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.07608 · Cited by 1 · see citing works
“We evaluate MiMo-7B-Base on a series of benchmarks, encompassing natural language understanding and reasoning, scientific question answering, reading comprehension, mathematics reasoning, coding, Chinese understanding, and long-context comprehension capabilities: Language understanding and reasoning : BBH (Suzgun et a…”
InfluentialSelf-citationThe Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs
Sinha, A., Arun, A., Goel, S., Staab, S., & Geiping, J. (2025) · arXiv · doi:10.48550/arxiv.2509.09677 · Cited by 1 · see citing works
“Where earlier we could only post-train on human demonstrations (Mishra et al., 2021), language models can now be trained with just rewards (Shao et al., 2024), enabling sophisticated reasoning (Guo et al., 2025; Jain et al., 2024) and agents (Kimi Team et al., 2025).”
Liu, S., Bao, L., Yang, Q., Geng, W., Zheng, B., Li, C., Chen, W., Peng, H., & Yuan, Y. (2026) · arXiv · doi:10.48550/arxiv.2602.03320 · Cited by 1 · see citing works
FormalAlign: Automated Alignment Evaluation for Autoformalization
Lu, J., Wan, Y., Huang, Y., Xiong, J., Liu, Z., & Guo, Z. (2024) · arXiv · doi:10.48550/arxiv.2410.10135 · Cited by 1 · see citing works
Memory in the Age of AI Agents
Hu, Y., Liu, S., Yue, Y., Zhang, G., Liu, B., Zhu, F., Lin, J., Guo, H., Dou, S., Xi, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2512.13564 · Cited by 1 · see citing works
From Reasoning to Code: GRPO Optimization for Underrepresented Languages
PENNINO, F., RAIMONDI, B., RONDELLI, M., GURIOLI, A., & GABBRIELLI, M. (2026) · Theory and Practice of Logic Programming · doi:10.1017/s1471068426100489 · Cited by 1 · see citing works
Yu, Y., Yao, L., Xie, Y., Tan, Q., Feng, J., Li, Y., & Wu, L. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.981 · Cited by 4 · see citing works
Self-citationDeepResearchGym: A Free, Transparent, and Reproducible Evaluation Sandbox for Deep Research
Coelho, J., Ning, J., He, J., Mao, K., Paladugu, A., Setlur, P., Jin, J., Callan, J., Magalhães, J., Martins, B., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.19253 · Cited by 1 · see citing works
“Link1, Link2, Link3, Link4, Link5, Link6, Link7, Link8, Link9, Link10 Why gas prices are so high Link1, Link2, Link3, Link4, Link5, Link6, Link7, Link8, Link9, Link10 Does religion cause war Link1, Link2, Link3, Link4, Link5, Link6, Link7, Link8, Link9, Link10”
RewardBench 2: Advancing Reward Model Evaluation
Malik, S., Pyatkin, V., Land, S., Morrison, J., Smith, N. A., Hajishirzi, H., & Lambert, N. (2025) · arXiv · doi:10.48550/arxiv.2506.01937 · Cited by 1 · see citing works
“The CoCoNot taxonomy does not always encourage outright refusal, but rather, rubrics are nuanced to allow for partial refusals where appropriate.”
InfluentialL’IA générative : une révolution technologique entre promesses et défis
Moulines, É. (2025) · Annales des Mines - Réalités industrielles · doi:10.3917/rindu1.252.0007 · Cited by 1 · see citing works
The green algorithm: can sustainability define the winner in the AI race?
Silva, S. R., Chinarro, D., & Prieto‐Andrés, A. (2025) · Frontiers in Political Science · doi:10.3389/fpos.2025.1629914 · Cited by 1 · see citing works
Yue, K., Peng, K., Lin, Y., Zhao, X., Xu, X., & Leung, V. C. M. (2025) · Ad Hoc Networks · doi:10.1016/j.adhoc.2025.104004 · Cited by 1 · see citing works
Artificial Intelligence at Seventy: From Symbolic Aspirations to Emergent Realities
Ochoa-Barragán, R., Raya-Tapia, A. Y., López-Flores, F. J., Márquez, C. R., & Ponce‐Ortega, J. M. (2026) · Industrial & Engineering Chemistry Research · doi:10.1021/acs.iecr.5c04476 · Cited by 1 · see citing works
Reasoning-Grounded Intent Injection for Generative Recommendation
Chen, X., Guo, P., Liu, F., Zhao, Y., Hu, Y., Que, M., Li, P., Wang, H., Fang, Z., Chen, W., et al. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808423 · Cited by 1 · see citing works
Reasoning BO: Enhancing Bayesian Optimization With the Reasoning Power of LLMs
Yang, Z., Wang, D., Ge, L., Wang, B., Fu, T., & Li, Y. (2026) · Materials Genome Engineering Advances · doi:10.1002/mgea.70086 · Cited by 3 · see citing works
Self-citationAdvance Toward Artificial Superintelligence with OpenAI's O1 Reinforcement Learning and Ethics
Mirindi, D., Sinkhonde, D., Mirindi, F., & Bezabih, T. (2025) · International Conference on Artificial Intelligence, Robotics and Control (AIRC) · doi:10.1109/airc64931.2025.11077494 · Cited by 1 · see citing works
Genome-Bench: A Scientific Reasoning Benchmark from Real-World Expert DiscussionsPreprint
Yin, M., Qu, Y., Liu, D., Yang, L., Cong, L., & Wang, M. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.06.02.657538 · Cited by 1 · see citing works
Auto-Formulating Dynamic Programming Problems with Large Language ModelsPreprint
Zhou, C., Jing-yuan, Y., Li, X., Chen, Y., He, Z., & Ge, D. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5350161 · Cited by 1 · see citing works
InfiMM-WebMath-40B: Advancing Multimodal Pre-Training for Enhanced Mathematical Reasoning
Han, X., Jian, Y., Hu, X., Liu, H., Wang, Y., Fan, Q., Ai, Y., Huang, H., He, R., Yang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2409.12568 · Cited by 1 · see citing works
Kong, Z., Li, Y., Zeng, F., Xin, L., Messica, S., Lin, X., Zhao, P., Kellis, M., Tang, H., & Zitnik, M. (2025) · arXiv · doi:10.48550/arxiv.2505.18227
“Additionally, integrating token reduction with training procedures like GRPO [116] is gaining traction; for instance, recent work reveals that optimizing only a subset of high-entropy ”forking tokens” matches full-gradient updates [140], suggesting that entropy patterns can effectively guide…”
Self-citationDetecting Data Contamination from Reinforcement Learning Post-training for Large Language Models
Tao, Y., Wang, T., Dong, Y., Liu, H., Zhang, K., Hu, X., & Li, G. (2025) · arXiv · doi:10.48550/arxiv.2510.09259 · Cited by 1 · see citing works
“The objective for a method like GRPO (Shao et al., 2024a) can be abstracted as: where f ( · ) is a function of the reward, the current policy, and a reference policy.”
InfluentialiCLP: Large Language Model Reasoning with Implicit Cognition Latent Planning
Chen, S., & Niu, D. (2025) · arXiv · doi:10.48550/arxiv.2512.24014
“While leveraging external knowledge bases can help mitigate these errors (Lyu et al., 2024; Zhu et al., 2024), accessing useful information from them is time-consuming, and many tasks lack effective knowledge bases altogether.”
InfluentialCritique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback
Zhang, X., Zhang, Y., Sun, H., Feng, K., Lu, C., Yang, C., & Meng, H. (2025) · arXiv · doi:10.48550/arxiv.2506.03106
“Many studies [11, 12, 13, 14, 15] primarily use critiques for evaluation, transforming them into numerical rewards for model improvement via RL algorithms such as Proximal Policy Optimization (PPO) [16] or Group Relative Policy Optimization (GRPO) [17].”
InfluentialRecMem: Recurrence-based Memory Consolidation for Efficient and Effective Long-Running LLM Agents
Dai, Z., Deng, S., Guan, S., Tian, Y., Yao, X., Yan, X., & Cheng, J. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.1619
“Large Language Models (LLMs) have demonstrated strong capabilities across a wide range of tasks (Guo et al., 2024; Shao et al., 2024).”
Just-In-Time Reinforcement Learning: Continual Learning in LLM Agents Without Gradient Updates
Li, Y., Lin, Z., Deng, A., Zhang, X., He, Y., Ji, S., Cao, T., & Hooi, B. (2026) · arXiv · doi:10.48550/arxiv.2601.18510
“(Shao et al., 2024) This baseline implements a gradient-centric Reinforcement Learning (RL) strategy to iteratively refine the agent’s policy.”
InfluentialSelf-citationDeshpande, D., Gangal, V., Mehta, H., Kannappan, A., Qian, R., & Wang, P. (2025) · arXiv · doi:10.48550/arxiv.2510.01353
“Reinforcement Environments for LLM Agents The recent shift towards better alignment techniques [56] has resulted in better generalization for different tasks[57].”
Self-citationSkin Tokens: A Learned Compact Representation for Unified Autoregressive Rigging
Zhang, J.-P., Pu, C.-F., Guo, M.-H., Cao, Y.-P., & Hu, S.-M. (2026) · arXiv · doi:10.48550/arxiv.2602.04805
“Unlike prior work that relies on costly annotated preference data (DPO) [Guo et al. 2025a; Rafailov et al. 2023], we leverage Group Relative Policy Optimization (GRPO) [Shao et al. 2024].”
Experience Compression Spectrum: Unifying Memory, Skills, and Rules in LLM Agents
Zhang, X., Wang, G., Cui, Y., Qiu, W., Li, Z., Zhu, B., & He, P. (2026) · arXiv
“Constitutional AI (Bai et al., 2022) uses pre-specified rules; reward design methods (PBRS (Ng et al., 1999), process rewards (Lightman et al., 2023), rule-based rewards (Shao et al., 2024)) encode L 3 -type knowledge but are human-designed, not learned.”
Sun, X., Wang, Q., Li, H., & Chen, L. (2026) · arXiv · doi:10.48550/arxiv.2608.03137
“The operation-normalized objective is (11) Equation (11) uses the clipped surrogate from PPO and candidate-group normalization from GRPO (Schulman et al., 2017; Shao et al., 2024).”
Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents
Patel, V. (2026) · arXiv
“We train with GRPO (Shao et al., 2024), which estimates its baseline from a group of sampled rollouts rather than a learned value critic (a natural fit for our programmatic, verifiable reward), on the prime-rl stack (LoRA, rank 32, α = 64 (Hu et al., 2022); learning rate 10 − 5 ; batch 32, group 8;…”
InfluentialHarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry
Chen, T., Lu, S., Zhao, K., Meng, W., Teng, H., Li, T., Li, C., Liu, X., Liang, J., Zhang, Z., et al. (2026) · arXiv
“We adopt Group Relative Policy Optimization (GRPO) [33].”
On Effectiveness and Efficiency of Agentic Tool-calling and RL Training
Liu, T., Qian, C., Cief, M., He, Y., Dan, D., Aletras, N., & Kazai, G. (2026) · arXiv
“Using GRPO as a representative algorithm, we identify two dominant sources of wasted computation in practice and propose lightweight fixes that reduce end-to-end training time without sacrificing performance.”
InfluentialContinuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live
Li, H., He, R., Mang, Q., Zhang, Q., Mao, H., Chen, X., Zhou, H., Zhang, H., Cheung, A., Gonzalez, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2511.02230
Self-citationMiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement
Core Team, Qiao, Z., Hua, Z., Ou, Z., Yue, Z., Jiang, Z., Huang, Z., Chen, Z., Zheng, Z., Xu, Z., et al. (2026) · arXiv · doi:10.48550/arxiv.2610.11959
Self-citationJust-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents
Zhou, Y., Li, Y., Liu, Z. L., Yavuz, S., & Joty, S. (2026) · arXiv · doi:10.48550/arxiv.2609.27334
Self-citationSWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training
Song, H., Huang, L., Sun, S., Jiang, J., Le, R., Cheng, D., Chen, G., Hu, Y., Chen, Z., Jia, Y., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.03411
“We adopt Reinforcement Learning with Verifiable Reward (RLVR) as our foundational paradigm, employing the Group Relative Policy Optimization (GRPO) algorithm [32] to optimize SWE-Master-SFT directly.”
ACTIVE-o3: Empowering MLLMs with Active Perception via Pure Reinforcement Learning
Zhu, M., Zhong, H., Zhao, C., Du, Z., Liu, M., Huang, Z., Li, A., Chen, H., Zou, C., Chen, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.21457
- Authors
- Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y. K., Wu, Y., & Guo, D.
- Affiliations
- DeepSeek (China) (CN)
- Peking University (CN)
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2402.03300
- OpenAlex
- W4391631327
- Cited by
- 101
- References
- 0
- Citations by year
- 2026: 49 · 2025: 47 · 2024: 5
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- synthetic environments, playworlds, preference optimization, entropy collapse, out-of-distribution evaluation, few-shot prompting, LLM agents
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
