The DOI matches its DataCite record; 1 of the 13 references it cites were found in a registry or an index, and 12 are not checked yet.
DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
Open accessAbstract
Mathematical reasoning poses a significant challenge for language models due to its complex and structured nature. In this paper, we introduce DeepSeekMath 7B, which continues pre-training DeepSeek-Coder-Base-v1.5 7B with 120B math-related tokens sourced from Common Crawl, together with natural language and code data. DeepSeekMath 7B has achieved an impressive score of 51.7% on the competition-level MATH benchmark without relying on external toolkits and voting techniques, approaching the performance level of Gemini-Ultra and GPT-4. Self-consistency over 64 samples from DeepSeekMath 7B achieves 60.9% on MATH. The mathematical reasoning capability of DeepSeekMath is attributed to two key factors: First, we harness the significant potential of publicly available web data through a meticulously engineered data selection pipeline. Second, we introduce Group Relative Policy Optimization (GRPO), a variant of Proximal Policy Optimization (PPO), that enhances mathematical reasoning abilities while concurrently optimizing the memory usage of PPO.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Recent studies primarily rely on rule-based verification (Shao et al., 2024; Mroueh, 2025), where agent trajectories are evaluated against predefined rules (Wei et al., 2025).”
From AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation (2026) · cites this paper as Shao et al., 2024
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Among recent advances, GRPO (Shao et al., 2024) enhances stability by optimizing policies based on the relative quality of sampled trajectories, removing the need for an explicit value function.”
From Agentic Memory: Learning Unified Long-Term and Short-Term Memory Management for Large Language Model Agents (2026) · cites this paper as Shao et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Early methods for mid-training data curation prioritized high-value sources by up-sampling domain-specific corpora and curated reasoning data, particularly those with CoT trajectories, to strengthen relevant analytical and reasoning capabilities in specific domains [30,176].”
From A Survey of Large Language Models (2026) · cites this paper as [30]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“State-of-the-art approaches [98, 2] typically employ reinforcement learning frameworks [116], where a multi-objective reward function is designed to optimize correctness and length compliance jointly.”
From Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality (2025) · cites this paper as [116]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Outcome-based RL like GRPO (Shao et al., 2024) addresses this by encouraging diverse reasoning paths, including error and self-correction, while given ground-truth feedback”
From Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models (2025) · cites this paper as Shao et al., 2024
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Reinforcement learning (RL) algorithms like PPO (Schulman et al., 2017) and GRPO (Shao et al., 2024) have effectively aligned LLM agents for complex tasks.”
From Just-In-Time Reinforcement Learning: Continual Learning in LLM Agents Without Gradient Updates (2026) · cites this paper as Shao et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The run halts itself at step 175 on ten consecutive zero-advantage batches: once almost every sampled rollout in a group answers correctly, GRPO's group-relative advantage (and hence the gradient) goes to zero.”
From Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents (2026) · cites this paper as Shao et al., 2024
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Our results indicate that RLVR with Group Region Policy Optimization (GRPO) [21] and data augmentation leads to significant performance increases on old IF benchmarks and IFBENCH.”
From Generalizing Verifiable Instruction Following (2025) · cites this paper as [21]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“it has been shown that it is also possible to perform RLVR training on a base model, leading to good math and reasoning performance [21, 29, 31].”
From Generalizing Verifiable Instruction Following (2025) · cites this paper as [21]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Reinforcement learning with verifiable rewards has been shown to be an effective approach for improving mathematical reasoning and coding abilities in language models [15, 21, 8, 33, 10, 13]”
From Generalizing Verifiable Instruction Following (2025) · cites this paper as [21]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“One notable example is GRPO [167], which eliminates the need for a separate value model, which group multiple rollouts for a single input prompt and uses the mean of rewards as a baseline.”
From RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs (2026) · cites this paper as [167]
The cited source says what the claim attributes to it.
In the citing paper
“Technically, capability internalization is achieved by learning from reasoning traces, through supervised fine-tuning (Wei et al., 2022; Zelikman et al., 2022; Schick et al., 2023; Mukherjee et al., 2023) or preference-guided optimization methods such as DPO (Rafailov et al., 2023; Tunstall et al., 2023; Yuan et al., 2024c; Grattafiori et al., 2024) and GRPO (Shao et al., 2024; DeepSeek-AI et al.,”
From Memory in the Age of AI Agents (2025) · cites this paper as Shao et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“This formulation avoids an explicit value function while maintaining PPO-style stability.”
From Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning (2025) · cites this paper as Shao et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“GRPO enables a streamlined approach to policy optimization, wherein the advantage function is derived by normalizing rewards within a cohort of responses generated for each prompt.”
From Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning (2025) · cites this paper as Shao et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“We experimented with various on-policy methods such as Group Relative Policy Optimization (GRPO) [66] and Proximal Policy Optimization (PPO) [67], but found the training to be unstable.”
From Self-Adapting Language Models (2025) · cites this paper as [66]
The cited source says what the claim attributes to it.
In the citing paper
“This can be viewed as an interpolation between per-question baselines, as in GRPO (Shao et al., 2024), and a global baseline, allowing for more structured variance reduction tailored to each task setup.”
From Absolute Zero: Reinforced Self-play Reasoning with Zero Data (2025) · cites this paper as Shao et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“Notably, DeepSeek-R1 (Guo et al., 2025) further demonstrates the feasibility of pure RL with Group Relative Policy Optimisation (Shao et al., 2024) when ground-truth verification is possible.”
From A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems (2025) · cites this paper as Shao et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“achieving performance competitive with GRPO (Shao et al., 2024), which relies on reinforcement learning.”
From iCLP: Large Language Model Reasoning with Implicit Cognition Latent Planning (2025) · cites this paper as Shao et al., 2024
The cited source says what the claim attributes to it.
In the citing paper
“In the normal mode and the more challenging cross mode , iCLP consistently outperforms all baselines and closely matches the performance of the GRPO algorithm (Shao et al., 2024).”
From iCLP: Large Language Model Reasoning with Implicit Cognition Latent Planning (2025) · cites this paper as Shao et al., 2024
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“non-thinking mode still exhibits the blind spot.”
From Self-Correction Bench: Uncovering and Addressing the Self-Correction Blind Spot in Large Language Models (2025) · cites this paper as Shao et al., 2024
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“held-out oracle accuracy rises from 9.0% to 16.7% (13/78 vs. 7/78 correct), a + 7.7-point ( + 86% relative) gain that nearly doubles the untrained baseline”
From Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents (2026) · cites this paper as Shao et al., 2024
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“On the procedural training distribution, GRPO reward climbs from 0.66 to 0.97 (Figure 4a).”
From Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents (2026) · cites this paper as Shao et al., 2024
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 41
- J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021. · arXiv
- Z. Azerbayev, H. Schoelkopf, K. Paster, M. D. Santos, S. McAleer, A. Q. Jiang, J. Deng, S. Bider- man, and S. Welleck. Llemma: An open language model for mathematics. arXiv preprint arXiv:2310.10631, 2023. · arXiv
- J. Bai, S. Bai, Y. Chu, Z. Cui, K. Dang, X. Deng, Y. Fan, W. Ge, Y. Han, F. Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023. · arXiv
- C. Burns, P. Izmailov, J. H. Kirchner, B. Baker, L. Gao, L. Aschenbrenner, Y. Chen, A. Ecoffet, M. Joglekar, J. Leike, et al. Weak-to-strong generalization: Eliciting strong capabilities with weak supervision. arXiv preprint arXiv:2312.09390, 2023. · arXiv
- M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba. Evaluating large language models trained on code. CoRR, abs/2107.03374, 2021. URL https://arxiv.org/abs/2107.03374. · arXiv
- K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. · arXiv
- Z. Du, Y. Qian, X. Liu, M. Ding, J. Qiu, Z. Yang, and J. Tang. Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 320-335, 2022.
- L. Gao, A. Madaan, S. Zhou, U. Alon, P. Liu, Y. Yang, J. Callan, and G. Neubig. PAL: program- aided language models. In A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett, editors, International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 10764-10799. PMLR, 2023. URL https://proceedings.mlr.press/v202/gao23f. html.
- D. Guo, Q. Zhu, D. Yang, Z. Xie, K. Dong, W. Zhang, G. Chen, X. Bi, Y. Wu, Y. K. Li, F. Luo, Y. Xiong, and W. Liang. Deepseek-coder: When the large language model meets programming -the rise of code intelligence, 2024. · DOI · Paper page
- D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020. · arXiv
- D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. Mea- suring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. · arXiv
- A. Q. Jiang, S. Welleck, J. P. Zhou, W. Li, J. Liu, M. Jamnik, T. Lacroix, Y. Wu, and G. Lample. Draft, sketch, and prove: Guiding formal theorem provers with informal proofs. arXiv preprint arXiv:2210.12283, 2022. · arXiv
- A. Joulin, E. Grave, P. Bojanowski, M. Douze, H. Jégou, and T. Mikolov. Fasttext. zip: Compress- ing text classification models. arXiv preprint arXiv:1612.03651, 2016. · arXiv
- W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
- Y. Leviathan, M. Kalman, and Y. Matias. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pages 19274-19286.
- PMLR, 2023.
- A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. Ramasesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, et al. Solving quantitative reasoning problems with language models. Advances in Neural Information Processing Systems, 35:3843-3857, 2022a.
- A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. V. Ramasesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, Y. Wu, B. Neyshabur, G. Gur-Ari, and V. Misra. Solving quantitative reasoning problems with language models. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 -December 9, 2022, 2022b. URL http://papers.nips. cc/paper_files/paper/2022/hash/18abbeef8cfe9203fdf9053c9c4fe191-Abstr act-Conference.html.
- H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe. Let's verify step by step. arXiv preprint arXiv:2305.20050, 2023. · arXiv
- I. Loshchilov and F. Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. · arXiv
- H. Luo, Q. Sun, C. Xu, P. Zhao, J. Lou, C. Tao, X. Geng, Q. Lin, S. Chen, and D. Zhang. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint arXiv:2308.09583, 2023. · arXiv
- X. Nguyen, W. Zhang, X. Li, M. M. Aljunied, Q. Tan, L. Cheng, G. Chen, Y. Deng, S. Yang, C. Liu, H. Zhang, and L. Bing. Seallms -large language models for southeast asia. CoRR, abs/2312.00738, 2023. doi: 10.48550/ARXIV.2312.00738. URL https://doi.org/10.485 50/arXiv.2312.00738. · DOI
- OpenAI. GPT4 technical report. arXiv preprint arXiv:2303.08774, 2023. · arXiv
- L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744, 2022.
- L. C. Paulson. Three years of experience with sledgehammer, a practical link between auto- matic and interactive theorem provers. In R. A. Schmidt, S. Schulz, and B. Konev, editors, Proceedings of the 2nd Workshop on Practical Aspects of Automated Reasoning, PAAR-2010, Edinburgh, Scotland, UK, July 14, 2010, volume 9 of EPiC Series in Computing, pages 1-10. EasyChair, 2010. doi: 10.29007/TNFD. URL https://doi.org/10.29007/tnfd. · DOI
- S. Polu and I. Sutskever. Generative language modeling for automated theorem proving. CoRR, abs/2009.03393, 2020. URL https://arxiv.org/abs/2009.03393. · arXiv
- J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015. · arXiv
- J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. · arXiv
- F. Shi, M. Suzgun, M. Freitag, X. Wang, S. Srivats, S. Vosoughi, H. W. Chung, Y. Tay, S. Ruder, D. Zhou, D. Das, and J. Wei. Language models are multilingual chain-of-thought reasoners. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https://openreview.net/pdf?id= fR3wGCk-IXp.
- F. Song, B. Yu, M. Li, H. Yu, F. Huang, Y. Li, and H. Wang. Preference ranking optimization for human alignment. arXiv preprint arXiv:2306.17492, 2023. · arXiv
- M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261, 2022. · arXiv
- T. H. Trinh, Y. Wu, Q. V. Le, H. He, and T. Luong. Solving olympiad geometry without human demonstrations. Nature, 625(7995):476-482, 2024.
- P. Wang, L. Li, L. Chen, F. Song, B. Lin, Y. Cao, T. Liu, and Z. Sui. Making large language models better reasoners with alignment. arXiv preprint arXiv:2309.02144, 2023a. · arXiv
- P. Wang, L. Li, Z. Shao, R. Xu, D. Dai, Y. Li, D. Chen, Y. Wu, and Z. Sui. Math-shepherd: Verify and reinforce llms step-by-step without human annotations. CoRR, abs/2312.08935, 2023b. · arXiv
- J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS, 2022. URL http://papers.nips.cc/paper_files/paper/2022/hash/9d5609613524ecf 4f15af0f7b31abca4-Abstract-Conference.html.
- H. Xia, T. Ge, P. Wang, S.-Q. Chen, F. Wei, and Z. Sui. Speculative decoding: Exploiting speculative execution for accelerating seq2seq generation. In H. Bouamor, J. Pino, and K. Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 3909- 3925, Singapore, Dec. 2023. Association for Computational Linguistics. doi: 10.18653/v1/20 · DOI
- H. Xia, Z. Yang, Q. Dong, P. Wang, Y. Li, T. Ge, T. Liu, W. Li, and Z. Sui. Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding. arXiv preprint arXiv:2401.07851, 2024. · arXiv
- S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y. Cao, and K. Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601, 2023. · arXiv
- Z. Yuan, H. Yuan, C. Li, G. Dong, C. Tan, and C. Zhou. Scaling relationship on learning mathematical reasoning with large language models. arXiv preprint arXiv:2308.01825, 2023a. · arXiv
- Z. Yuan, H. Yuan, C. Tan, W. Wang, S. Huang, and F. Huang. Rrhf: Rank responses to align language models with human feedback without tears. arXiv preprint arXiv:2304.05302, 2023b. · arXiv
- K. Zheng, J. M. Han, and S. Polu. Minif2f: a cross-system benchmark for formal olympiad-level mathematics. arXiv preprint arXiv:2109.00110, 2021. · arXiv
Cited by
Sorted by citations
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
Self-citationDeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al. (2025) · Nature · doi:10.1038/s41586-025-09422-z · Cited by 1,701
Self-citationLarge Language Models for Information Retrieval: A Survey
Zhu, Y., Yuan, H., Wang, S., Liu, J., Liu, W., Deng, C., Chen, H., Liu, Z., Dou, Z., & Wen, J.-R. (2026) · ACM Transactions on Information Systems · doi:10.1145/3748304 · Cited by 225
Graph Retrieval-Augmented Generation: A Survey
Peng, B., Zhu, Y., Liu, Y., Bo, X., Shi, H., Hong, C., Zhang, Y., & Tang, S. (2026) · ACM Transactions on Information Systems · doi:10.1145/3777378 · Cited by 206
Generative artificial intelligence in medicine
Teo, Z. L., Thirunavukarasu, A. J., Elangovan, K., Cheng, H., Moova, P., Soetikno, B., Nielsen, C., Pollreisz, A., Ting, D. S. J., Morris, R. J. T., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03983-2 · Cited by 119
Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.09388 · Cited by 17,283
Self-citationExploring the role of large language models in the scientific method: from hypothesis to discovery
Zhang, Y., Khan, S. A., Mahmud, A., Yang, H., Lavin, A., Levin, M. G., Frey, J. G., Dunnmon, J., Evans, J., Bundy, A., et al. (2025) · npj Artificial Intelligence · doi:10.1038/s44387-025-00019-5 · Cited by 160
RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs
Chaudhari, S., Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K., Deshpande, A., & Castro da Silva, B. (2026) · ACM Computing Surveys · doi:10.1145/3743127 · Cited by 92
Agentic Large Language Models, a Survey
Plaat, A., Van Duijn, M., Van Stein, N., Preuss, M., Van der Putten, P., & Batenburg, K. J. (2025) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.18675 · Cited by 75
Toward large reasoning models: A survey of reinforced reasoning with large language models
Xu, F., Hao, Q., Shao, C., Zong, Z., Li, Y., Wang, J., Zhang, Y., Wang, J., Lan, X., Gong, J., et al. (2025) · Patterns · doi:10.1016/j.patter.2025.101370 · Cited by 533
Self-citationRahman, A., Mahir, S. H., Tashrif, M. T. A., Karim, M. A., Aishi, A. A., Kundu, D., Debnath, T., Moududi, M. A. A., Eidmum, M. Z. A., Miah, A. S. M., et al. (2025) · Systems and Soft Computing · doi:10.1016/j.sasc.2025.200396 · Cited by 101
Shojaee, P., Mirzadeh, I., Alizadeh, K., Horton, M., Bengio, S., & Farajtabar, M. (2025) · arXiv · doi:10.48550/arxiv.2506.06941 · Cited by 45
Scaling Large Language Models for Next-Generation Single-Cell AnalysisPreprint
Rizvi, S. A., Levine, D., Patel, A. M., Zhang, S., Wang, E., Perry, C. J., Vrkic, I., Constante, N. M., Fu, Z., He, S., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.04.14.648850 · Cited by 54
Aydın, Ö., Karaarslan, E., Erenay, F. S., & Bačanin, N. (2026) · Turkish Journal of Engineering · doi:10.31127/tuje.1838765 · Cited by 69
Reasoning beyond limits: Advances and open problems for LLMs
Ferrag, M. A., Tihanyi, N., & Debbah, M. A. (2025) · ICT Express · doi:10.1016/j.icte.2025.09.003 · Cited by 56
Critique of impure reason: Unveiling the reasoning behaviour of medical large language models
Sim, S. Z. Y., & Chen, T. (2025) · eLife · doi:10.7554/elife.106187 · Cited by 29
DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
DeepSeek-AI, Liu, A., Mei, A., Lin, B., Xue, B., Wang, B., Xu, B., Wu, B., Zhang, B., Lin, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2512.02556 · Cited by 19
“2, we still adopt Group Relative Policy Optimization (GRPO) (DeepSeek-AI, 2025; Shao et al., 2024) as the RL training algorithm.”
Self-citationTransforming agency: On the mode of existence of large language models
Barandiaran, X. E., & Almendros, L. S. (2025) · Phenomenology and the Cognitive Sciences · doi:10.1007/s11097-025-10094-3 · Cited by 35
Training Large Language Models to Reason in a Continuous Latent Space
Hao, S., Sukhbaatar, S., Su, D., Li, X., Hu, Z., Weston, J., & Tian, Y. (2024) · arXiv · doi:10.48550/arxiv.2412.06769 · Cited by 18
“…prompting LLMs (Wei et al., 2022; Khot et al., 2022; Zhou et al., 2022), or training LLMs to generate reasoning chains, either with supervised finetuning (Yue et al., 2023; Yu et al., 2023) or reinforcement learning (Wang et al., 2024; Havrilla et al., 2024; Shao et al., 2024; Yu et al., 2024a).”
SciCode: A Research Coding Benchmark Curated by Scientists
Tian, M., Gao, L., Zhang, S. D., Chen, X., Fan, C., Guo, X., Haas, R., Ji, P., Krongchon, K., Li, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.13168 · Cited by 15
Self-citationAbsolute Zero: Reinforced Self-play Reasoning with Zero Data
Zhao, A., Wu, Y., Yue, Y., Wu, T., Xu, Q., Yue, Y., Lin, M., Wang, S., Wu, Q., Zheng, Z., et al. (2025) · arXiv · Cited by 13
Self-citationZheng, X., Yang, J., Zhou, L., Qin, L., Lu, C., Alonso‐Vante, N., Huang, Z., Zhang, P., Zhou, Y., Luo, Z.-H., et al. (2025) · Chemical Engineering Science · doi:10.1016/j.ces.2025.121740 · Cited by 16
Wang, S., Yu, L., Gao, C., Zheng, C., Liu, S., Lu, R., Dang, K., Chen, X., Yang, J., Zhang, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.01939 · Cited by 14
“The estimated advantage ˆ A it is then computed as: , where In addition to this modified advantage estimation, GRPO adds a KL penalty term to the clipped objective in Equation (2).”
InfluentialLarge Language Models for Combinatorial Optimization: A Systematic Review
Ros, F. D., Soprano, M., Gaspero, L. D., & Roitero, K. (2026) · ACM Computing Surveys · doi:10.1145/3801961 · Cited by 40
Wang, P., Holmes, J., Liu, Z., Chen, D., Liu, T., Shen, J., & Liu, W. (2025) · Frontiers in Oncology · doi:10.3389/fonc.2025.1557064 · Cited by 13
Self-citationFunctional alignment of protein language models via reinforcement learning
Blalock, N., Seshadri, S., Nakamura, K., Babbar, A., Fahlberg, S. A., Kulkarni, A., & Romero, P. A. (2026) · Nature Communications · doi:10.1038/s41467-026-77557-2 · Cited by 19
The Hallucination Tax of Reinforcement Finetuning
Song, L., Shi, T., & Zhao, J. (2025) · Findings of the Association for Computational Linguistics: EMNLP 2025 · doi:10.18653/v1/2025.findings-emnlp.112 · Cited by 10
Luo, X., Huang, Y., Cui, J., & Zheng, K. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130795 · Cited by 14
General-Purpose Models for the Chemical Sciences: LLMs and Beyond
Alampara, N., Aneesh, A., Ríos-García, M., Mirza, A., Schilling-Wilhelmi, M., Aghajani, A. A., Sun, M., Prastalo, G., & Jablonka, K. M. (2026) · Chemical Reviews · doi:10.1021/acs.chemrev.5c00583 · Cited by 24
A Survey of Post-Training Scaling in Large Language Models
Lai, H., Liu, X., Gao, J., Cheng, J., Qi, Z., Xu, Y., Yao, S., Zhang, D., Du, J., Hou, Z., et al. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.140 · Cited by 8
GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
Agrawal, L. A., Tan, S., Soylu, D., Ziems, N., Khare, R., Opsahl-Ong, K., Singhvi, A., Shandilya, H., Ryan, M. J., Jiang, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2507.19457 · Cited by 7
Jin, S., Fu, Z., Song, G., Pei, J., Gong, H., Ma, L., & Zhang, D. (2025) · Corrosion Science · doi:10.1016/j.corsci.2025.113217 · Cited by 8
Zhu, Y., Shu, X., Liu, X., Li, Y., Yi, B., & Ma, D. (2025) · Anesthesiology and Perioperative Science · doi:10.1007/s44254-025-00156-9 · Cited by 9
Self-citationBenchmarking Benchmark Leakage in Large Language Models
Xu, R., Wang, Z., Fan, R.-Z., & Liu, P. (2024) · arXiv · doi:10.48550/arxiv.2404.18824 · Cited by 6
Self-citationFang, J., Peng, Y., Zhang, X., Wang, Y., Yi, X., Zhang, G., Xu, Y., Wu, B., Liu, S., Li, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.07407 · Cited by 6
Ce‐LLMs: Status and trends of education‐specific large language models developed in China
Xie, T., Zhou, Y., & Yu, J. (2025) · Future in Educational Research · doi:10.1002/fer3.70008 · Cited by 7
DeepSeek in Education: Exploring the Transformative Potential of AI‐Driven Educational Intelligence
Liao, J., Sun, F., Liu, Y., & Hu, Y. (2025) · Future in Educational Research · doi:10.1002/fer3.70022 · Cited by 6
Agentic Lab: An Agentic-physical AI system for cell and organoid experimentation and manufacturingPreprint
Wang, W., Swain, S., Lee, J., Lin, Z., Canales, B., Aljović, A., Liu, Y., Li, Q., Marin‐Llobet, A., Liu, M., et al. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.11.11.686354 · Cited by 10
Quantifying artificial intelligence through algorithmic generalization
Ito, T., Campbell, M. S., Lior Horesh, C., Klinger, T., & Ram, P. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01092-w · Cited by 10
DeepSeek: implications for data science and management in the AI era
Xu, Z., Liu, S., Huang, W., Shi, J., Xu, F., Tang, X., & Lu, H. (2025) · Data Science and Management · doi:10.1016/j.dsm.2025.09.001 · Cited by 6
WisWheat: A Three-Tiered Vision-Language Dataset for Wheat Management
Yuan, B., Song, S., Fernández, J. A., Luo, Y., Baktashmotlagh, M., & Huang, Z. H. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3758256 · Cited by 5
HPC-R1: Characterizing R1-like Large Reasoning Models on HPC
Weingram, A., Zhang, D., Chen, Z., Qi, H., & Lu, X. (2025) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3712285.3759827 · Cited by 6
Awan, S. A., Khan, M. A., Khan, A. A., Sathio, A. A., Alsayaydeh, J. A. J., Bacarra, R., Herawan, S. G., & Aziz, R. (2025) · Journal Of Big Data · doi:10.1186/s40537-025-01330-3 · Cited by 7
Synthetic4Health: generating annotated synthetic clinical letters
Ren, L., Belkadi, S., Han, L., Del-Pinto, W., & Nenadić, G. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1497130 · Cited by 12
LIMO: Less is More for Reasoning
Ye, Y., Huang, Z., Xiao, Y., Chern, E., Xia, S., & Liu, P. (2025) · arXiv · doi:10.48550/arxiv.2502.03387 · Cited by 4
“Two key developments have created conditions for reimagining reasoning in LLMs: (1) Knowledge foundation revolution : Modern foundation models now incorporate unprecedented amounts of mathematical content during pre-training (Qwen et al., 2025; Yang et al., 2024; Wang et al., 2024).”
de Almeida, V. A., de Camargo, V., Gómez-Bravo, R., van Boven, K., van der Haring, E., Finger, M., & López, L. F. (2026) · JAMIA Open · doi:10.1093/jamiaopen/ooag017 · Cited by 5
Exploring Text-to-Motion Generation with Human Preference
Sheng, J., Lin, M. G., Zhao, A., Pruvost, K., Wen, Y., Li, Y., Huang, G. M., & Liu, Y. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00195 · Cited by 8
Self-citationE2ETune: End-to-End Knob Tuning via Fine-Tuned Generative Language Model
Huang, X., Li, H., Zhang, J., Zhao, X., Yao, Z., Li, Y., Zhang, T., Chen, J., Chen, H., & Li, C. (2025) · Proceedings of the VLDB Endowment · doi:10.14778/3773731.3773732 · Cited by 31
Self-citationEnhancing math reasoning ability of large language models via computation logic graphs
Zhao, D., Han, D., Wu, J. X., He, Z., Ning, B., Yuan, Y., Li, Y., Wang, C., & Song, S. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113905 · Cited by 4
Self-citationGeneralizing Verifiable Instruction Following
Pyatkin, V., Malik, S., Graf, V., Ivison, H., Huang, S., Dasigi, P., Lambert, N., & Hajishirzi, H. (2025) · arXiv · doi:10.48550/arxiv.2507.02833 · Cited by 4
Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
Yue, Y., Chen, Z., Lu, R., Zhao, A., Wang, Z., Yue, Y., Song, S., & Huang, G. (2025) · arXiv · doi:10.48550/arxiv.2504.13837 · Cited by 804
“Recently, Reinforcement Learning with Verifiable Rewards (RLVR) has gained significant traction as a method to improve the reasoning capabilities of LLMs in domains such as mathematics and programming (Lambert et al., 2024; Shao et al., 2024).”
InfluentialZweiger, A., Pari, J., Guo, H., Akyürek, E., Kim, Y., & Agrawal, P. (2025) · arXiv · Cited by 3
“We experimented with various on-policy methods such as Group Relative Policy Optimization (GRPO) [66] and Proximal Policy Optimization (PPO) [67], but found the training to be unstable.”
Large Language Model Reasoning Failures
Song, P., Han, P., & Goodman, N. (2026) · arXiv · doi:10.48550/arxiv.2602.06176 · Cited by 3
Gao, H.-A., Geng, J., Hua, W., Hu, M., Juan, X., Liu, H., Liu, S., Qiu, J., Qi, X., Wu, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2507.21046 · Cited by 3
Self-citationSeed-Coder: Let the Code Model Curate Data for Itself
Seed, B., Zhang, Y., Su, J., Sun, Y., Xi, C., Xiao, X., Zheng, S., Zhang, A., Liu, K., Zan, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.03524 · Cited by 3
“During the RL training process, similar to DeepCoder [Luo et al., 2025], we used the open-source verl framework [Sheng et al., 2025] for GRPO training [Shao et al., 2024], and adopted optimization techniques similar to DAPO [Yu et al., 2025, ByteDance Seed, 2025].”
Self-citationStop Overthinking: A Survey on Efficient Reasoning for Large Language Models
Sui, Y., Chuang, Y.-N., Wang, G., Zhang, J., Zhang, T., Yuan, J., Liu, H., Wen, A., Zhong, S., Zou, N., et al. (2025) · arXiv · Cited by 517
Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory
Long, L., He, Y., Ye, W., Pan, Y., Lin, Y., Li, H., Zhao, J., & Li, W. (2025) · arXiv · doi:10.48550/arxiv.2508.09736 · Cited by 102
“We also use Group Relative Policy Optimization (GRPO)[37] to optimize the policy model in the ablation study.”
Yan, S., Yang, X., Huang, Z., Nie, E., Ding, Z., Li, Z., Ma, X., Bi, J., Kersting, K., Pan, J. Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2508.19828 · Cited by 297
“Both agents are fine-tuned using PPO (Schul-man et al., 2017) or GRPO (Shao et al., 2024), achieving strong performance with as few as 152 question–answer pairs.”
Demystifying Multi-Agent Debate: The Role of Confidence and Diversity
Zhu, X., Zhang, C., Chi, Y., Stafford, T., Collier, N., & Vlachos, A. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.1694 · Cited by 28
When DeepSeek-R1 meets financial applications: benchmarking, opportunities, and limitations
Liu, S., Chen, L., Yan, J., Jiang, Y., Wang, X., Li, X., & Yang, Q. (2025) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2500227 · Cited by 2
Leveraging label semantics and meta-label refinement for multi-label question classification
Dong, S., Niu, X., Zhong, R., Wang, Z., & Zuo, M. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113412 · Cited by 2
VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations for Synthetic VideosPreprint
Li, Z., Wu, X., Qin, Y., Shi, G., Du, H., Manocha, D., Zhou, T., & Boyd-Graber, J. L. (2025) · Qeios · doi:10.32388/bxc6x1 · Cited by 2
Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning
NVIDIA, :, Azzolini, A., Bai, J., Brandon, H., Cao, J., Chattopadhyay, P., Chen, H., Chu, J., Cui, Y., et al. (2025) · arXiv · doi:10.48550/arxiv.2503.15558 · Cited by 187
Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination
Wu, M., Zhang, Z., Dong, Q., Xi, Z., Zhao, J., Jin, S., Fan, X., Zhou, Y., Lv, H., Zhang, M., et al. (2025) · arXiv · doi:10.48550/arxiv.2507.10532 · Cited by 75
“However, these methods fails to transfer to Llama or OLMo (OLMo et al. 2025), suggesting model-specific idiosyncrasies.”
Self-citationOh, G., Kim, S., Park, S., & Kim, B.-H. (2026) · Journal of Medical Internet Research · doi:10.2196/90693 · Cited by 7
“Med-R1 [32] uses group relative policy optimization [23] to improve generalizability across various medical imaging modalities, achieving a 29.94% accuracy improvement.”
InfluentialMiMo: Unlocking the Reasoning Potential of Language Model - From Pretraining to Posttraining
Xiaomi, L.-C., :, Xia, B., Shen, B., Cici, Zhu, D., Zhang, D., Wang, G., Zhang, H., Liu, H., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.07608 · Cited by 1
“We evaluate MiMo-7B-Base on a series of benchmarks, encompassing natural language understanding and reasoning, scientific question answering, reading comprehension, mathematics reasoning, coding, Chinese understanding, and long-context comprehension capabilities: Language understanding and reasoning : BBH (Suzgun et a…”
InfluentialSelf-citationThe Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs
Sinha, A., Arun, A., Goel, S., Staab, S., & Geiping, J. (2025) · arXiv · doi:10.48550/arxiv.2509.09677 · Cited by 1
“Where earlier we could only post-train on human demonstrations (Mishra et al., 2021), language models can now be trained with just rewards (Shao et al., 2024), enabling sophisticated reasoning (Guo et al., 2025; Jain et al., 2024) and agents (Kimi Team et al., 2025).”
Liu, S., Bao, L., Yang, Q., Geng, W., Zheng, B., Li, C., Chen, W., Peng, H., & Yuan, Y. (2026) · arXiv · doi:10.48550/arxiv.2602.03320 · Cited by 1
FormalAlign: Automated Alignment Evaluation for Autoformalization
Lu, J., Wan, Y., Huang, Y., Xiong, J., Liu, Z., & Guo, Z. (2024) · arXiv · doi:10.48550/arxiv.2410.10135 · Cited by 1
Memory in the Age of AI Agents
Hu, Y., Liu, S., Yue, Y., Zhang, G., Liu, B., Zhu, F., Lin, J., Guo, H., Dou, S., Xi, Z., et al. (2025) · arXiv · doi:10.48550/arxiv.2512.13564 · Cited by 389
From Reasoning to Code: GRPO Optimization for Underrepresented Languages
PENNINO, F., RAIMONDI, B., RONDELLI, M., GURIOLI, A., & GABBRIELLI, M. (2026) · Theory and Practice of Logic Programming · doi:10.1017/s1471068426100489 · Cited by 1
Yu, Y., Yao, L., Xie, Y., Tan, Q., Feng, J., Li, Y., & Wu, L. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.981 · Cited by 4
Self-citationDeepResearchGym: A Free, Transparent, and Reproducible Evaluation Sandbox for Deep Research
Coelho, J., Ning, J., He, J., Mao, K., Paladugu, A., Setlur, P., Jin, J., Callan, J., Magalhães, J., Martins, B., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.19253 · Cited by 1
“Link1, Link2, Link3, Link4, Link5, Link6, Link7, Link8, Link9, Link10 Why gas prices are so high Link1, Link2, Link3, Link4, Link5, Link6, Link7, Link8, Link9, Link10 Does religion cause war Link1, Link2, Link3, Link4, Link5, Link6, Link7, Link8, Link9, Link10”
RewardBench 2: Advancing Reward Model Evaluation
Malik, S., Pyatkin, V., Land, S., Morrison, J., Smith, N. A., Hajishirzi, H., & Lambert, N. (2025) · arXiv · doi:10.48550/arxiv.2506.01937 · Cited by 1
“The CoCoNot taxonomy does not always encourage outright refusal, but rather, rubrics are nuanced to allow for partial refusals where appropriate.”
InfluentialL’IA générative : une révolution technologique entre promesses et défis
Moulines, É. (2025) · Annales des Mines - Réalités industrielles · doi:10.3917/rindu1.252.0007 · Cited by 1
The green algorithm: can sustainability define the winner in the AI race?
Silva, S. R., Chinarro, D., & Prieto‐Andrés, A. (2025) · Frontiers in Political Science · doi:10.3389/fpos.2025.1629914 · Cited by 1
Yue, K., Peng, K., Lin, Y., Zhao, X., Xu, X., & Leung, V. C. M. (2025) · Ad Hoc Networks · doi:10.1016/j.adhoc.2025.104004 · Cited by 1
Artificial Intelligence at Seventy: From Symbolic Aspirations to Emergent Realities
Ochoa-Barragán, R., Raya-Tapia, A. Y., López-Flores, F. J., Márquez, C. R., & Ponce‐Ortega, J. M. (2026) · Industrial & Engineering Chemistry Research · doi:10.1021/acs.iecr.5c04476 · Cited by 1
Reasoning-Grounded Intent Injection for Generative Recommendation
Chen, X., Guo, P., Liu, F., Zhao, Y., Hu, Y., Que, M., Li, P., Wang, H., Fang, Z., Chen, W., et al. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3808423 · Cited by 1
Reasoning BO: Enhancing Bayesian Optimization With the Reasoning Power of LLMs
Yang, Z., Wang, D., Ge, L., Wang, B., Fu, T., & Li, Y. (2026) · Materials Genome Engineering Advances · doi:10.1002/mgea.70086 · Cited by 13
Self-citationAdvance Toward Artificial Superintelligence with OpenAI's O1 Reinforcement Learning and Ethics
Mirindi, D., Sinkhonde, D., Mirindi, F., & Bezabih, T. (2025) · International Conference on Artificial Intelligence, Robotics and Control (AIRC) · doi:10.1109/airc64931.2025.11077494 · Cited by 1
Genome-Bench: A Scientific Reasoning Benchmark from Real-World Expert DiscussionsPreprint
Yin, M., Qu, Y., Liu, D., Yang, L., Cong, L., & Wang, M. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.06.02.657538 · Cited by 2
Auto-Formulating Dynamic Programming Problems with Large Language ModelsPreprint
Zhou, C., Jing-yuan, Y., Li, X., Chen, Y., He, Z., & Ge, D. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5350161 · Cited by 16
InfiMM-WebMath-40B: Advancing Multimodal Pre-Training for Enhanced Mathematical Reasoning
Han, X., Jian, Y., Hu, X., Liu, H., Wang, Y., Fan, Q., Ai, Y., Huang, H., He, R., Yang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2409.12568 · Cited by 1
Kong, Z., Li, Y., Zeng, F., Xin, L., Messica, S., Lin, X., Zhao, P., Kellis, M., Tang, H., & Zitnik, M. (2025) · arXiv · doi:10.48550/arxiv.2505.18227 · Cited by 51
“Additionally, integrating token reduction with training procedures like GRPO [116] is gaining traction; for instance, recent work reveals that optimizing only a subset of high-entropy ”forking tokens” matches full-gradient updates [140], suggesting that entropy patterns can effectively guide…”
Self-citationDetecting Data Contamination from Reinforcement Learning Post-training for Large Language Models
Tao, Y., Wang, T., Dong, Y., Liu, H., Zhang, K., Hu, X., & Li, G. (2025) · arXiv · doi:10.48550/arxiv.2510.09259 · Cited by 1
“The objective for a method like GRPO (Shao et al., 2024a) can be abstracted as: where f ( · ) is a function of the reward, the current policy, and a reference policy.”
InfluentialiCLP: Large Language Model Reasoning with Implicit Cognition Latent Planning
Chen, S., & Niu, D. (2025) · arXiv · doi:10.48550/arxiv.2512.24014
“While leveraging external knowledge bases can help mitigate these errors (Lyu et al., 2024; Zhu et al., 2024), accessing useful information from them is time-consuming, and many tasks lack effective knowledge bases altogether.”
InfluentialCritique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback
Zhang, X., Zhang, Y., Sun, H., Feng, K., Lu, C., Yang, C., & Meng, H. (2025) · arXiv · doi:10.48550/arxiv.2506.03106 · Cited by 92
“Many studies [11, 12, 13, 14, 15] primarily use critiques for evaluation, transforming them into numerical rewards for model improvement via RL algorithms such as Proximal Policy Optimization (PPO) [16] or Group Relative Policy Optimization (GRPO) [17].”
InfluentialRecMem: Recurrence-based Memory Consolidation for Efficient and Effective Long-Running LLM Agents
Dai, Z., Deng, S., Guan, S., Tian, Y., Yao, X., Yan, X., & Cheng, J. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.1619
“Large Language Models (LLMs) have demonstrated strong capabilities across a wide range of tasks (Guo et al., 2024; Shao et al., 2024).”
Just-In-Time Reinforcement Learning: Continual Learning in LLM Agents Without Gradient Updates
Li, Y., Lin, Z., Deng, A., Zhang, X., He, Y., Ji, S., Cao, T., & Hooi, B. (2026) · arXiv · doi:10.48550/arxiv.2601.18510
“(Shao et al., 2024) This baseline implements a gradient-centric Reinforcement Learning (RL) strategy to iteratively refine the agent’s policy.”
InfluentialSelf-citationDeshpande, D., Gangal, V., Mehta, H., Kannappan, A., Qian, R., & Wang, P. (2025) · arXiv · doi:10.48550/arxiv.2510.01353
“Reinforcement Environments for LLM Agents The recent shift towards better alignment techniques [56] has resulted in better generalization for different tasks[57].”
Self-citationSkin Tokens: A Learned Compact Representation for Unified Autoregressive Rigging
Zhang, J.-P., Pu, C.-F., Guo, M.-H., Cao, Y.-P., & Hu, S.-M. (2026) · arXiv · doi:10.48550/arxiv.2602.04805
“Unlike prior work that relies on costly annotated preference data (DPO) [Guo et al. 2025a; Rafailov et al. 2023], we leverage Group Relative Policy Optimization (GRPO) [Shao et al. 2024].”
Experience Compression Spectrum: Unifying Memory, Skills, and Rules in LLM Agents
Zhang, X., Wang, G., Cui, Y., Qiu, W., Li, Z., Zhu, B., & He, P. (2026) · arXiv
“Constitutional AI (Bai et al., 2022) uses pre-specified rules; reward design methods (PBRS (Ng et al., 1999), process rewards (Lightman et al., 2023), rule-based rewards (Shao et al., 2024)) encode L 3 -type knowledge but are human-designed, not learned.”
Sun, X., Wang, Q., Li, H., & Chen, L. (2026) · arXiv · doi:10.48550/arxiv.2608.03137
“The operation-normalized objective is (11) Equation (11) uses the clipped surrogate from PPO and candidate-group normalization from GRPO (Schulman et al., 2017; Shao et al., 2024).”
Supersede: Diagnosing and Training the Memory-Update Gap in LLM Agents
Patel, V. (2026) · arXiv
“We train with GRPO (Shao et al., 2024), which estimates its baseline from a group of sampled rollouts rather than a learned value critic (a natural fit for our programmatic, verifiable reward), on the prime-rl stack (LoRA, rank 32, α = 64 (Hu et al., 2022); learning rate 10 − 5 ; batch 32, group 8;…”
InfluentialHarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry
Chen, T., Lu, S., Zhao, K., Meng, W., Teng, H., Li, T., Li, C., Liu, X., Liang, J., Zhang, Z., et al. (2026) · arXiv
“We adopt Group Relative Policy Optimization (GRPO) [33].”
On Effectiveness and Efficiency of Agentic Tool-calling and RL Training
Liu, T., Qian, C., Cief, M., He, Y., Dan, D., Aletras, N., & Kazai, G. (2026) · arXiv
“Using GRPO as a representative algorithm, we identify two dominant sources of wasted computation in practice and propose lightweight fixes that reduce end-to-end training time without sacrificing performance.”
InfluentialContinuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live
Li, H., He, R., Mang, Q., Zhang, Q., Mao, H., Chen, X., Zhou, H., Zhang, H., Cheung, A., Gonzalez, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2511.02230
Self-citationMiMo-V2.6: Scaling Reinforcement Learning Towards Self-Improvement
Core Team, Qiao, Z., Hua, Z., Ou, Z., Yue, Z., Jiang, Z., Huang, Z., Chen, Z., Zheng, Z., Xu, Z., et al. (2026) · arXiv · doi:10.48550/arxiv.2610.11959 · Cited by 9
Self-citationJust-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents
Zhou, Y., Li, Y., Liu, Z. L., Yavuz, S., & Joty, S. (2026) · arXiv · doi:10.48550/arxiv.2609.27334
Self-citation
Metadata
- Authors
- Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y. K., Wu, Y., & Guo, D.
- Affiliations
- DeepSeek (China) (CN)
- Peking University (CN)
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2402.03300
- OpenAlex
- W4391631327
- Cited by
- 101
- References
- 0
- Citations by year
- 2026: 49 · 2025: 47 · 2024: 5
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- synthetic environments, playworlds, preference optimization, entropy collapse, out-of-distribution evaluation, few-shot prompting, LLM agents
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
