The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning

Agarwal, Shivam, Zhang, Zimin, Yuan, Lifan, Han, Jiawei, Peng, Hao·2025·arXiv
DOI 10.48550/arxiv.2505.15134

Cited by (148)

Items 101–148 of 148

  1. Two is better than one: A Collapse-free Multi-Reward RLIF Training Framework
    Shourov Joarder, Diganta Sikdar, Ahsan Habib Akash, Binod Bhattarai, Prashnna Gyawali · 2026 · arXiv (Cornell University)
  2. Smart Picks in the Dark: Towards Efficient RLVR for Reasoning via Tracing Metacognitive Pivots
    G. Zhu, Shenzhi Yang, Haobo Wang, Xing Zheng, Yingfan Ma, Xuening Feng, Zhongqi Chen, Bowen Song, Weiqiang Wang, Gang Chen · 2026 · arXiv (Cornell University)
  3. STAPO: Selective Trajectory-Aware Policy Optimization for LLM Agent Training
    Qiuyi Qi, Tian Liang, Mutian Bao, Jinjian Zhang, Dongnan Liu, Wei Zhou, Linjian Mo, Ming Kong, Jie Liu, F Zhang et al. · 2026 · arXiv (Cornell University)
  4. Calibrated e-CUSUM Decoding for Quantized Reasoning Models: Why Token Log-Probability Is the Wrong Observable for Decoding Monitors
    El Hassane Ettifouri, AYOUB BELFATMI, Mahaman Sanoussi Yahaya Alassan, Walid Dahhane · 2026 · arXiv (Cornell University)
  5. CUSUM-Shaped Inference-Time Monitoring and Targeted Re-Decoding for Quantized Small Language Model Reasoning
    El Hassane Ettifouri, AYOUB BELFATMI, Mahaman Sanoussi Yahaya Alassan, Walid Dahhane · 2026 · arXiv (Cornell University)
  6. Self-Improving LLM Agents at Test-Time
    Acikgoz, Emre Can, Qian, Cheng, Ji, Heng, Hakkani-Tür, Dilek, Tur, Gokhan · 2025 · arXiv
  7. Unsupervised Elicitation of Language Models
    Wen, Jiaxin, Ankner, Zachary, Somani, Arushi, Hase, Peter, Marks, Samuel, Goldman-Wetzler, Jacob, Petrini, Linda, Sleight, Henry, Burns, Collin, He, He et al. · 2025 · arXiv
  8. Breaking the Self-Confirming Loop: Diagnosing and Mitigating Systemic Reward Bias in Self-Rewarding RL
    Tan, Chuyi, Yuan, Peiwen, Wang, Xinglin, Li, Yiwei, Feng, Shaoxiong, Zhang, Yueqi, Shi, Jiayi, Zhang, Ji, Pan, Boyuan, Hu, Yao et al. · 2025 · arXiv
  9. Why Self-Rewarding Works: Theoretical Guarantees for Iterative Alignment of Language Models
    Fu, Shi, Wang, Yingjie, Hu, Shengchao, Wang, Peng, Tao, Dacheng · 2026 · arXiv
  10. A beautiful loop: An active inference theory of consciousness
    Ruben Laukkonen, Karl John Friston, Shamil Chandaria · 2025 · PsyArXiv (OSF Preprints)
  11. Computational Stylometrics and the Pauline Corpus: Limits in Authorship Attribution
    Anthony Rosa · 2025 · Religions
  12. GEM: Generative Entropy-Guided Preference Modeling for Few-shot Alignment of LLMs
    Zhao, Yiyang, Bai, Huiyu, Xuejiao Zhao · 2025 · arXiv (Cornell University)
  13. Neural Signatures of Predictive Processing during Natural Mandarin Speech Comprehension
    Qifei Wang, Jakub Szewczyk, Judit Fazekas, Eva Berlot, Floris P. de Lange · 2025 · bioRxiv (Cold Spring Harbor Laboratory)
  14. A Survey on Entropy Mechanism in Large Reasoning Models
    Zihan Wang, Xiatinghan Xu, Hao Wang, Yiwen Ye, Yuchen Li, Linhao Wang, Hongze Tan, Peidong Wang, Feng Shi, Guoqing Chen et al. · 2026 · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)
  15. PaperGuide: Making Small Language-Model Paper-Reading Agents More Efficient
    Zijian Wang, Tiancheng Huang, Hanqi Li, Da Ma, Lu Chen, Kai Yu · 2026 · arXiv (Cornell University)
  16. Unsupervised Elicitation of Moral Values from Language Models
    Meysam Alizadeh, Fabrizio Gilardi, Zeynab Samei · 2026 · Proceedings of IASEAI Conference
  17. Toward IIT-Inspired Consciousness in LLMs: A Reward-Based Learning Framework
    Hamid Reza Akbari, M. Sameti, Amir M. Mansourian, Mohammad Hossein Rohban, Hossein Sameti · 2026 · arXiv (Cornell University)
  18. Med-Evo: Test-time Self-evolution for Medical Multimodal Large Language Models
    Dunyuan Xu, Xikai Yang, Juzheng Miao, Yaoqian Li, Jinpeng Li, Pheng-Ann Heng · 2026 · arXiv (Cornell University)
  19. Model Whisper: Steering Vectors Unlock Large Language Models’ Potential in Test-Time
    Xinyue Kang, Diwei Shi, Li Jia Chen · 2026 · Proceedings of the AAAI Conference on Artificial Intelligence
  20. GEM: Generative Entropy-Guided Preference Modeling for Few-Shot Alignment of LLMs
    Yiyang Zhao, Bai Huiyu, Xuejiao Zhao · 2026 · Proceedings of the AAAI Conference on Artificial Intelligence
  21. When and Why Does Unsupervised RL Succeed in Mathematical Reasoning? A Manifold Envelopment Perspective
    Zelin Zhang, Fei Cheng, Chenhui Chu · 2026 · arXiv (Cornell University)
  22. Rethinking Entropy Minimization in Test-Time Adaptation for Autoregressive Models
    Wei-Ping Huang, Chee-En Yu, Guan-Ting Lin, Hung-yi Lee · 2026 · Interspeech (USB)
  23. Label-Free Reinforcement Learning via Cross-Model Entropy
    Matt Gorbett, Hossein Mozdorani Shirazi · 2026 · arXiv (Cornell University)
  24. Fine-Tuning Improves Information Conveyance in Language Models
    Yuwei Cheng, Weiyi Tian, Haifeng Xu · 2026 · arXiv (Cornell University)
  25. Beyond Surface Alignment: Grounding the Dynamics of Situational Understanding and Generative Control in LLMs
    Chenghao Yang · 2026 · arXiv (Cornell University)
  26. Easy Samples Are All You Need: Self-Evolving LLMs via Data-Efficient Reinforcement Learning
    Zhiyin Yu, Bo Zhang, Qibin Hou, Zhonghai Wu, Xiao Luo, Lei Bai · 2026 · Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
  27. A Survey of Reinforcement Learning for Large Language Models under Data Scarcity: Challenges and Solutions
    Zhiyin Yu, Yuchen Mou, Juncheng Yan, Junyu Luo, Chunchun Chen, Xing Wei, Yunhui Liu, Hongru Sun, Yuxing Zhang, Jun Xu et al. · 2026 · Proceedings of the conference - Association for Computational Linguistics. Meeting
  28. Verifier-Free RL for LLMs via Intrinsic Gradient-Norm Reward
    Xuexiang Wen, Hang Yu, Linchao Zhu, Gaoang Wang · 2026 · Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
  29. Free Energy-Driven Reinforcement Learning with Adaptive Advantage Shaping for Unsupervised Reasoning in LLMs
    Yiming Huang, Zhenbo Shi, Xin-Cheng Wen, Jichuan Zeng, Cuiyun Gao, Peiyi Han, Chuanyi Liu · 2026 · Proceedings of the conference - Association for Computational Linguistics. Meeting
  30. Entropy Scheduling in Reinforcement Learning for Large Language Models
    Xingjin Wang, Howe Tissue, Lu Wang, Linjing Li, Daniel Dajun Zeng · 2026 · Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
  31. Not All Citations Are Equal:Entropy-Guided Citation Selection for Noise-Resistant Medical LLM
    Minyu Gao, Hanlin Xiao, Ruoyu Wang, Shuai Yang, YeXuan Zhang, Xin Wu, Xingyu Liu · 2026 · Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
  32. Concept rather than Document: Context Compression via AMR-based Conceptual Entropy
    Kaize Shi, Xueyao Sun, Xiaohui Tao, Li Lin, Qika Lin, Guandong Xu · 2026 · Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
  33. C3D: Enhancing LLM Reasoning via Confidence-Guided Contrastive Decoding
    Yufeng Zhang, Xuepeng Wang, Lingxiang Wu, Jinqiao Wang · 2026 · Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
  34. Think Earlier, Not Longer: Prompt Optimization via Reducing Unhealthy Exploration
    Ling-I Wu, Minyu Chen, Jingyang Li, Xi Chang, Ge Li · 2026 · Findings of the Association for Computational Linguistics: ACL (Findings of ACL)
  35. DE-Venus: A Data-Efficient RLVR Framework for Large Language Models
    Shenzhi Yang, G. Zhu, Kai Tang, Zhengqing Zang, Xing Zheng, Haobo Wang, Yingfan Ma, Bowen Song, Bo Han, Bo An et al. · 2026 · arXiv (Cornell University)
  36. Balancing Accuracy and Confidence in LLMs: Reinforcement Learning with Entropy-Calibrated Rewards
    Keizo Kato, Chenhui Chu, Yugo Murawaki, Sadao Kurohashi · 2026 · Lecture notes in networks and systems
  37. Reinforcing Agentic Creativity in Scientific Ideation with Night Science
    Priyanka Kargupta, Silviu Cucerzan, Shweti Mahajan, Allen Herring, Jiawei Han, Ryen W. White, Sujay Kumar Jauhar · 2026 · arXiv (Cornell University)
  38. Certainty Is Not Just Correctness: Rethinking Token-Level Certainty in LLM Reasoning
    Yunfan Zhou, Ye Zhu, Zhihai Wang, Jianguo Yao, Haibing Guan, Xijun Li · 2026 · arXiv (Cornell University)
  39. Sharpen Before You Adapt: Data-Free Entry-State Sharpening for Test-Time Reinforcement Learning
    Zhanming Zhang, Vinoth Selvendran · 2026 · arXiv (Cornell University)
  40. Sharpen Without Search: On-Policy Distillation of Sequence-Level Power Distribution
    Erfan Baghaei Potraghloo, Seyedarmin Azizi, Arya Fayyazi, Saeid Shokoufa, Mehdi Kamal, Souvik Kundu, Massoud Pedram · 2026 · arXiv (Cornell University)
  41. Unlocking Token Rewards via Training-Free Reward Attribution
    Sitong Wu, 谈浩如, Bin Xia, Xichen Zhang, Jingyao Li, Shaofeng Zhang, Xiaojuan Qi, Bei Yu, Jiaya Jia · 2026 · arXiv (Cornell University)
  42. Preserving the SPINE: How a Few Critical Tokens Prevent Global Entropy Collapse in LLM Reasoning
    Jaeeun Jang, Hansle Lee, Sangmin Kim · 2026 · International Joint Conference on Neural Networks (IJCNN)
  43. What RL Brings to Reasoning: A Few Decisions the Base Model Would Not Make
    Siyu Zhang, Yuquan Lu, Yoshua Y. Li, Yifan Ding, Kun Zeng, Yu Huo, Yifu Guo, Rongxiang Weng, Xunliang Cai, Xiaoying Tang · 2026 · arXiv (Cornell University)
  44. What Outcome RL Buys: Finishing, Reliability, Rarely Reach
    Shan Yang · 2026 · arXiv (Cornell University)
  45. Consistency-Calibrated Advantage Estimation for Label-Free Reinforcement Learning
    Meng Yang, Yongle Zhao, Zhiyang Zhang, Xiaorui Jiang, Peirou Liang, Furui Chen, Xie Yin, Ziyong Feng · 2026 · Conference on Empirical Methods in Natural Language Processing (EMNLP)
  46. When Reward-Free Signals Are Not Directions: A Gradient-Level Diagnostic for Test-Time Reasoning Optimization
    Shijie Zheng, Yinxing Dai, Zhiqiang Pu · 2026 · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)
  47. TTML: Test-Time Memory Learning for Label-Free RLVR
    Yongshi Ye, Biao Fu, Xinjie Chen, Minpeng Liao, Xiaodong Shi · 2026 · Conference on Empirical Methods in Natural Language Processing (EMNLP)
  48. There Is No Free Lunch in RLVR: Every pass@1 Gain Has a pass@k Price
    Yinuo Liu · 2026 · arXiv (Cornell University)