The DOI matches its DataCite record; 1 of the 2 references it cites were found in a registry or an index, and 1 are not checked yet.
O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning
Open accessAbstract
Recently, long-thought reasoning LLMs, such as OpenAI's O1, adopt extended reasoning processes similar to how humans ponder over complex problems. This reasoning paradigm significantly enhances the model's problem-solving abilities and has achieved promising results. However, long-thought reasoning process leads to a substantial increase in inference time. A pressing challenge is reducing the inference overhead of long-thought LLMs while ensuring accuracy. In this paper, we experimentally demonstrate that long-thought reasoning models struggle to effectively allocate token budgets based on problem difficulty and reasoning redundancies. To address this, we propose Length-Harmonizing Fine-Tuning (O1-Pruner), aiming at minimizing reasoning overhead while maintaining accuracy. This effective fine-tuning method first estimates the LLM's baseline performance through pre-sampling and then uses RL-style fine-tuning to encourage the model to generate shorter reasoning processes under accuracy constraints. This allows the model to achieve efficient reasoning with lower redundancy while maintaining accuracy. Experiments on various mathematical reasoning benchmarks show that O1-Pruner not only significantly reduces inference overhead but also achieves higher accuracy, providing a novel and promising solution to this challenge. Our code is coming soon at https://github.com/StarDewXXX/O1-Pruner
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Reinforcement-learning-driven token reduction has shown strong promise for improving reasoning efficiency in both LLMs and MLLMs.
“Reinforcement-learning (RL)-driven token reduction has shown strong promise for improving reasoning efficiency in both LLMs and MLLMs.”
From Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality (2025) · cites this paper as [98]
The Abstract reports that RL-style fine-tuning shortened LLM reasoning and reduced inference overhead on mathematical benchmarks. The Introduction mentions multimodal reasoning, but does not say that RL-driven token reduction improved efficiency in MLLMs. The claim's extension to MLLMs is therefore unsupported by this paper. (LLM, conf=0.87)
Experimental — accuracy not yet measuredThe key challenge for reinforcement-learning-driven token reduction is to balance compute and reasoning quality via dynamic reward design, sparsity-inducing constraints, and adaptive control of effective token length.
“The key challenge is to balance compute and reasoning quality via dynamic reward design, sparsity-inducing constraints, and adaptive control of effective token length [2, 98, 151].”
From Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality (2025) · cites this paper as [98]
The Introduction and §4.2 describe RL-based fine-tuning that rewards shorter reasoning while protecting accuracy and encourages shorter answers for simpler problems. The paper does not describe sparsity-inducing constraints; its stated constraint concerns accuracy. It defines a length-harmonizing reward, but does not describe dynamically redesigning that reward. (LLM, conf=0.86)
Experimental — accuracy not yet measuredState-of-the-art approaches for controllable reasoning typically employ reinforcement learning frameworks, where a multi-objective reward function is designed to optimize correctness and length compliance jointly.
“State-of-the-art approaches [98, 2] typically employ reinforcement learning frameworks [116], where a multi-objective reward function is designed to optimize correctness and length compliance jointly.”
From Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality (2025) · cites this paper as [98]
The Introduction and §4.2 describe O1-Pruner as an RL-based method with a reward that balances shorter reasoning against accuracy. They do not establish that state-of-the-art approaches typically use this framework, and the paper rewards shorter outputs relative to a reference rather than compliance with a specified length. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 17
- Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. 2024. Do not think that much for 2+3=? on the overthinking of o1-like llms. Preprint, arXiv:2412.21187. · arXiv
- Jeffrey Cheng and Benjamin Van Durme. 2024. Compressed chain of thought: Efficient reason- ing through dense representations. Preprint, arXiv:2412.13171. · arXiv
- Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word prob- lems. Preprint, arXiv:2110.14168. · arXiv
- Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. 2024b. Parameter-efficient fine- tuning for large models: A comprehensive survey. Preprint, arXiv:2403.14608. · arXiv
- Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. 2024. Training large language models to reason in a continuous latent space. Preprint, arXiv:2412.06769. · arXiv
- Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. Preprint, arXiv:2106.09685. · arXiv
- Yu Kang, Xianghui Sun, Liangyu Chen, and Wei Zou. 2024. C3ot: Generating shorter chain-of-thought without compromising effectiveness. Preprint, arXiv:2412.11664. · arXiv
- Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Ef- ficient memory management for large language model serving with pagedattention. Preprint, arXiv:2309.06180. · arXiv
- Yu Meng, Mengzhou Xia, and Danqi Chen. 2024. Simpo: Simple preference optimization with a reference-free reward. Preprint, arXiv:2405.14734. · arXiv
- Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2024. Direct preference optimization: Your lan- guage model is secretly a reward model. Preprint, arXiv:2305.18290. · arXiv
- John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Prox- imal policy optimization algorithms. Preprint, arXiv:1707.06347. · arXiv
- Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. Preprint, arXiv:2402.03300. · arXiv
- Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Ku- mar. 2024. Scaling llm test-time compute optimally can be more effective than scaling model parameters. Preprint, arXiv:2408.03314. · arXiv
- Kimi Team, Angang Du, Bofei Gao, and et al. 2025. Kimi k1.5: Scaling reinforcement learning with llms. Preprint, arXiv:2501.12599. · arXiv
- Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. 2024. Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models. Preprint, arXiv:2408.00724. · arXiv
- Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, and 1 others. 2024. Mulberry: Empowering mllm with o1-like reasoning and reflection via collective monte carlo tree search. arXiv preprint arXiv:2412.18319. · arXiv
- Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. 2024. Marco-o1: Towards open rea- soning models for open-ended solutions. Preprint, arXiv:2411.14405. · arXiv
Cited by
Sorted by citations
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
Sui, Y., Chuang, Y.-N., Wang, G., Zhang, J., Zhang, T., Yuan, J., Liu, H., Wen, A., Zhong, S., Zou, N., et al. (2025) · arXiv · Cited by 517
“.5 [83]; O1-Pruner [57]; L1 [1]; Training [3]; Demystifying [104]; DAST [74]; MRT [68]; Stepwise [23]; SFT with Variable-Length CoT e.g. Distilling 2-1 [105]; C3oT [40]; TokenSkip [95]; CoT-Valve [59]; Self-Training [62]; Learn to Skip [53]; Token-Budget [34]; ReasoningOutput-based Latent e.g.…”
InfluentialKong, Z., Li, Y., Zeng, F., Xin, L., Messica, S., Lin, X., Zhao, P., Kellis, M., Tang, H., & Zitnik, M. (2025) · arXiv · doi:10.48550/arxiv.2505.18227 · Cited by 51
“The key challenge is to balance compute and reasoning quality via dynamic reward design, sparsity-inducing constraints, and adaptive control of effective token length [2, 98, 151].”
Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models
Liu, R., Sun, Y., Zhang, M., Bai, H., Yu, X., Yu, T., Yuan, C., & Hou, L. (2025) · arXiv · doi:10.48550/arxiv.2504.04823
“For reasoning models trained with reinforcement learning, a common way is to add a length penalty term on the reward in addition to the accuracy (Luo et al., 2025; Aggarwal & Welleck, 2025; Team et al., 2025).”
AdaDiff-VLA: Hybrid Reasoning via Mode Policy Learning for Vision-Language-Action Models
Liu, H., Liu, P., Liu, X., & Shu, T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38410-2_11
AdaThinking- : One-Token Entropy Regulation for Adaptive Thinking
Wang, Z., Guan, T., Chen, B., Guo, Z., Liu, J., Jin, C., Duan, C., Zhou, K., Yan, P., Shen, W., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37032-7_28
Metadata
- Authors
- Luo, H., Shen, L., He, H., Wang, Y., Liu, S., Li, W., Tan, N., Cao, X., & Tao, D.
- Affiliations
- Sun Yat-sen University (CN)
- Tsinghua University (CN)
- University of Oxford (GB)
- Nanyang Technological University (SG)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- Pages
- 14242-14257
- arXiv
- 2501.12570
- OpenAlex
- W4406772739
- Cited by
- 2
- References
- 0
- Citations by year
- 2026: 2
- FWCI
- 17.09
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Findings of the Association for Computational Linguistics: ACL (Findings of ACL) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Natural Science Foundation of China (62576364, JCYJ20241202124430041)
- Shenzhen Fundamental Research Program
- Basic and Applied Basic Research Foundation of Guangdong Province (2026B1515020071)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- inference efficiency, reinforcement learning fine-tuning, mathematical reasoning, length estimation, OpenAI o1
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
