CoChat
Sign Up
PreprintCited by 1Topic ModelingMultimodal Machine Learning ApplicationsNatural Language Processing Techniques

The DOI matches its DataCite record; 2 of the 2 references it cites were found in a registry or an index.

Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning

Open access
Chen, Xinghao, Zhao, Anhao, Xia, Heming, Lu, Xuan, Wang, Hanlin, Chen, Yanjun, Zhang, Wei, Wang, Jian, Li, Wenjie, Shen, Xiaoyu·2025·arXiv
DOI 10.48550/arxiv.2505.16782arXiv 2505.16782
Open this paper in CoChat — free accountOpen PDF

Abstract

Large Language Models (LLMs) have shown impressive performance on complex tasks through Chain-of-Thought (CoT) reasoning. However, conventional CoT relies on explicitly verbalized intermediate steps, which constrains its broader applicability, particularly in abstract reasoning tasks beyond language. To address this, there has been growing research interest in \textit{latent CoT reasoning}, where the reasoning process is embedded within latent spaces. By decoupling reasoning from explicit language generation, latent CoT offers the promise of richer cognitive representations and facilitates more flexible, faster inference. This paper aims to present a comprehensive overview of this emerging paradigm and establish a systematic taxonomy. We analyze recent advances in methods, categorizing them from token-wise horizontal approaches to layer-wise vertical strategies. We then provide in-depth discussions of these methods, highlighting their design principles, applications, and remaining challenges. We hope that our survey provides a structured foundation for advancing this promising direction in LLM reasoning. The relevant papers will be regularly updated at https://github.com/EIT-NLP/Awesome-Latent-CoT.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2505.16782

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

  • Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models

    Sui, Y., Chuang, Y.-N., Wang, G., Zhang, J., Zhang, T., Yuan, J., Liu, H., Wen, A., Zhong, S., Zou, N., et al. (2025) · arXiv · Cited by 517 · cited 1×

    “First, it introduces computational inefficiency (Lin et al., 2025b; Feng et al., 2025; Qu et al., 2025; Sui et al., 2025; Wang et al., 2025a; Liu et al., 2025), as not all tokens in the articulated thought process carry informative content.”

  • LatentEvolve: Self-Evolving Test-Time Scaling in Latent Space

    Zhang, G., Meng, F., Wan, G., Li, Z., Wang, K., Yin, Z., Bai, L., & Yan, S. (2025) · arXiv · doi:10.48550/arxiv.2509.24771 · cited 5×

    “…processes , as their internal reasoning processes occur in latent spaces that are not directly interpretable by humans (Lindsey et al., 2025); (2) evaluation gaps , with no clear metrics to distinguish deep latent reasoning from input-output shortcuts (Ameisen et al., 2025); and (3)…”
    Influential

Show all 2 references

Research works printed but not linked: 89

  1. Jannik Brinkmann, Abhay Sheshadri, Victor Levoso, Paul Swoboda, and Christian Bartelt. 2024. A mech- anistic analysis of a transformer trained on a sym- bolic multi-step reasoning task. In Findings of the Association for Computational Linguistics: ACL 2024, pages 4082-4102, Bangkok, Thailand. Associ- ation for Computational Linguistics. · DOI
  2. Mingyang Chen, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z. Pan, Wen Zhang, Huajun Chen, Fan Yang, Zenan Zhou, and Weipeng Chen. 2025a. Research: Learning to rea- son with search for llms via reinforcement learning. Preprint, arXiv:2503.19470. · arXiv
  3. Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. 2025b. Towards reasoning era: A survey of long chain-of-thought for reasoning large language models. Preprint, arXiv:2503.09567. · arXiv
  4. Runjin Chen, Zhenyu Zhang, Junyuan Hong, Souvik Kundu, and Zhangyang Wang. 2025c. Seal: Steer- able reasoning calibration of large language models for free. Preprint, arXiv:2504.07986. · arXiv
  5. Xingwu Chen and Difan Zou. 2024. What can trans- former learn with varying depth? case studies on sequence learning tasks. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org.
  6. Yanda Chen, Joe Benton, Ansh Radhakrishnan, Jonathan Uesato, Carson Denison, John Schulman, Arushi Somani, Peter Hase, Misha Wagner, Fabien Roger, Vlad Mikulik, Samuel R. Bowman, Jan Leike, Jared Kaplan, and Ethan Perez. 2025d. Reasoning models don't always say what they think. Preprint, arXiv:2505.05410. · arXiv
  7. Yilong Chen, Junyuan Shang, Zhenyu Zhang, Yanxi Xie, Jiawei Sheng, Tingwen Liu, Shuohuan Wang, Yu Sun, Hua Wu, and Haifeng Wang. 2025e. In- ner thinking transformer: Leveraging dynamic depth scaling to foster adaptive internal thinking. Preprint, arXiv:2502.13842. · arXiv
  8. Jeffrey Cheng and Benjamin Van Durme. 2024. Com- pressed chain of thought: Efficient reasoning through dense representations. Preprint, arXiv:2412.13171. · arXiv
  9. Zheng Chu, Jingchang Chen, Qianglong Chen, Wei- jiang Yu, Tao He, Haotian Wang, Weihua Peng, Ming Liu, Bing Qin, and Ting Liu. 2024. Navi- gate through enigmatic labyrinth a survey of chain of thought reasoning: Advances, frontiers and future. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1173-1203, Bangkok, Thai- land. Association for Computational Linguistics. · DOI
  10. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word prob- lems. Preprint, arXiv:2110.14168. · arXiv
  11. Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Lukasz Kaiser. 2019. Uni- versal transformers. In International Conference on Learning Representations.
  12. Yuntian Deng, Yejin Choi, and Stuart Shieber. 2024. From explicit cot to implicit cot: Learning to inter- nalize cot step by step. Preprint, arXiv:2405.14838. · arXiv
  13. Yuntian Deng, Kiran Prasad, Roland Fernandez, Paul Smolensky, Vishrav Chaudhary, and Stuart Shieber. 2023. Implicit chain of thought reasoning via knowl- edge distillation. Preprint, arXiv:2311.01460. · arXiv
  14. Sicheng Feng, Gongfan Fang, Xinyin Ma, and Xinchao Wang. 2025. Efficient reasoning models: A survey. Preprint, arXiv:2504.10903. · arXiv
  15. Jonas Geiping, Sean McLeish, Neel Jain, John Kirchen- bauer, Siddharth Singh, Brian R. Bartoldson, Bhavya Kailkhura, Abhinav Bhatele, and Tom Goldstein. 2025. Scaling up test-time compute with latent reasoning: A recurrent depth approach. Preprint, arXiv:2502.05171. · arXiv
  16. Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346- 361. · DOI · Paper page
  17. Zhuocheng Gong, Jian Guan, Wei Wu, Huishuai Zhang, and Dongyan Zhao. 2025. Latent preference coding: Aligning large language models via discrete latent codes. Preprint, arXiv:2505.04993. · arXiv
  18. Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Kr- ishna Menon, Sanjiv Kumar, and Vaishnavh Na- garajan. 2024. Think before you speak: Train- ing language models with pause tokens. In The Twelfth International Conference on Learning Representations.
  19. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: In- centivizing reasoning capability in llms via reinforce- ment learning. Preprint, arXiv:2501.12948. · arXiv
  20. Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. 2024. Training large language models to reason in a contin- uous latent space. Preprint, arXiv:2412.06769. · arXiv
  21. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021a. Measuring massive multitask language under- standing. In International Conference on Learning Representations.
  22. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021b. Measuring mathe- matical problem solving with the MATH dataset. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
  23. David Herel and Tomas Mikolov. 2024. Think- ing tokens for language modeling. Preprint, arXiv:2405.08644. · arXiv
  24. Yifan Hou, Jiaoda Li, Yu Fei, Alessandro Stolfo, Wangchunshu Zhou, Guangtao Zeng, Antoine Bosse- lut, and Mrinmaya Sachan. 2023. Towards a mech- anistic interpretation of multi-step reasoning capa- bilities of language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4902-4919, Singapore. Association for Computational Linguistics. · DOI
  25. Zemin Huang, Zhiyang Chen, Zijun Wang, Tiancheng Li, and Guo-Jun Qi. 2025. Reinforcing the diffu- sion chain of lateral thought with diffusion language models. Preprint, arXiv:2505.10446. · arXiv
  26. Yoichi Ishibashi, Taro Yano, and Masafumi Oyamada. 2025. Mining hidden thoughts from texts: Evaluat- ing continual pretraining with synthetic data for llm reasoning. Preprint, arXiv:2505.10182. · arXiv
  27. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar- Lezama, Koushik Sen, and Ion Stoica. 2025. Live- codebench: Holistic and contamination free evalu- ation of large language models for code. In The Thirteenth International Conference on Learning Representations.
  28. Yifan Ji, Zhipeng Xu, Zhenghao Liu, Yukun Yan, Shi Yu, Yishan Li, Zhiyuan Liu, Yu Gu, Ge Yu, and Maosong Sun. 2025. Learning more effective rep- resentations for dense retrieval through deliberate thinking before search. Preprint, arXiv:2502.12974. · arXiv
  29. Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. 2024. SWE-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Representations.
  30. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025a. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. Preprint, arXiv:2503.09516. · arXiv
  31. Mingyu Jin, Weidi Luo, Sitao Cheng, Xinyi Wang, Wenyue Hua, Ruixiang Tang, William Yang Wang, and Yongfeng Zhang. 2025b. Disentangling mem- ory and reasoning ability in large language models. Preprint, arXiv:2411.13504. · arXiv
  32. Takeshi Kojima, Shixiang (Shane) Gu, Machel Reid, Yu- taka Matsuo, and Yusuke Iwasawa. 2022. Large lan- guage models are zero-shot reasoners. In Advances in Neural Information Processing Systems, vol- ume 35, pages 22199-22213.
  33. Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi, Shusaku Sone, Masaya Taniguchi, Ana Brassard, Keisuke Sakaguchi, and Kentaro Inui. 2025. Think-to-talk or talk-to-think? when llms come up with an an- swer in multi-step arithmetic reasoning. Preprint, arXiv:2412.01113. · arXiv
  34. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. Albert: A lite bert for self-supervised learn- ing of language representations. In International Conference on Learning Representations.
  35. Yuanchun Li, Hao Wen, Weijun Wang, Xiangyu Li, Yizhen Yuan, Guohong Liu, Jiacheng Liu, Wenxing Xu, Xiang Wang, Yi Sun, Rui Kong, Yile Wang, Hanfei Geng, Jian Luan, Xuefeng Jin, Zilong Ye, Guanjing Xiong, Fan Zhang, Xiang Li, Mengwei Xu, Zhijun Li, Peng Li, Yang Liu, Ya-Qin Zhang, and Yunxin Liu. 2024. Personal llm agents: Insights and survey about the capability, efficiency and security. Preprint, arXiv:2401.05459. · arXiv
  36. Tianhe Lin, Jian Xie, Siyu Yuan, and Deqing Yang. 2025a. Implicit reasoning in transformers is reason- ing through shortcuts. Preprint, arXiv:2503.07604. · arXiv
  37. Zicheng Lin, Tian Liang, Jiahao Xu, Qiuzhi Lin, Xing Wang, Ruilin Luo, Chufan Shi, Siheng Li, Yujiu Yang, and Zhaopeng Tu. 2025b. Criti- cal tokens matter: Token-level contrastive estima- tion enhances llm's reasoning capability. Preprint, arXiv:2411.19943. · arXiv
  38. Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blun- som. 2017. Program induction by rationale gen- eration: Learning to solve and explain algebraic word problems. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 158- 167, Vancouver, Canada. Association for Compu- tational Linguistics. · DOI · Paper page
  39. Tengxiao Liu, Qipeng Guo, Xiangkun Hu, Cheng Ji- ayang, Yue Zhang, Xipeng Qiu, and Zheng Zhang. 2024a. Can language models learn to skip steps? In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
  40. Tianqiao Liu, Zui Chen, Zitao Liu, Mi Tian, and Weiqi Luo. 2024b. Expediting and elevating large language model reasoning via hidden chain-of-thought decod- ing. Preprint, arXiv:2409.08561. · arXiv
  41. Yue Liu, Jiaying Wu, Yufei He, Hongcheng Gao, Hongyu Chen, Baolong Bi, Jiaheng Zhang, Zhiqi Huang, and Bryan Hooi. 2025. Efficient inference for large reasoning models: A survey. Preprint, arXiv:2503.23077. · arXiv
  42. Ziqiao Ma, Jacob Sansom, Run Peng, and Joyce Chai. 2023. Towards a holistic landscape of situated theory of mind in large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1011-1031, Singapore. Association for Computational Linguistics. · DOI
  43. Shen-yun Miao, Chao-Chun Liang, and Keh-Yih Su. 2020. A diverse corpus for evaluating and developing English math word problem solvers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 975-984, Online. Association for Computational Linguistics. · DOI · Paper page
  44. Amirkeivan Mohtashami, Matteo Pagliardini, and Mar- tin Jaggi. 2025. CoTFormer: A chain of thought driven architecture with budget-adaptive computation cost at inference. In The Thirteenth International Conference on Learning Representations.
  45. Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021. Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2080-2094, Online. Association for Computational Linguistics. · DOI · Paper page
  46. Jacob Pfau, William Merrill, and Samuel R. Bowman. 2024. Let's think dot by dot: Hidden computation in transformer language models. In First Conference on Language Modeling.
  47. Xiaoye Qu, Yafu Li, Zhaochen Su, Weigao Sun, Jianhao Yan, Dongrui Liu, Ganqu Cui, Daizong Liu, Shuxian Liang, Junxian He, Peng Li, Wei Wei, Jing Shao, Chaochao Lu, Yue Zhang, Xian-Sheng Hua, Bowen Zhou, and Yu Cheng. 2025. A survey of efficient rea- soning for large reasoning models: Language, multi- modality, and beyond. Preprint, arXiv:2503.21614. · arXiv
  48. David Rein, Betty Li Hou, Asa Cooper Stickland, Jack- son Petty, Richard Yuanzhe Pang, Julien Dirani, Ju- lian Michael, and Samuel R. Bowman. 2024. GPQA: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling.
  49. Yangjun Ruan, Neil Band, Chris J. Maddison, and Tat- sunori Hashimoto. 2025. Reasoning to learn from latent thoughts. Preprint, arXiv:2503.18866. · arXiv
  50. Abulhair Saparov and He He. 2023. Language models are greedy reasoners: A systematic formal analysis of chain-of-thought. In The Eleventh International Conference on Learning Representations.
  51. Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, and Sashank J. Reddi. 2025a. Reasoning with latent thoughts: On the power of looped trans- formers. In The Thirteenth International Conference on Learning Representations.
  52. Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, and Sashank J. Reddi. 2025b. Reasoning with latent thoughts: On the power of looped trans- formers. Preprint, arXiv:2502.17416. · arXiv
  53. Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, and Sashank J. Reddi. 2025c. Reasoning with latent thoughts: On the power of looped trans- formers. In The Thirteenth International Conference on Learning Representations.
  54. Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, Furong Huang, Uzi Vishkin, Micah Goldblum, and Tom Goldstein. 2021. Can you learn an algorithm? gen- eralizing from easy to hard problems with recur- rent networks. In Advances in Neural Information Processing Systems.
  55. Yuval Shalev, Amir Feder, and Ariel Goldstein. 2024. Distributional reasoning in llms: Parallel reason- ing processes in multi-hop reasoning. Preprint, arXiv:2406.13858. · arXiv
  56. Xuan Shen, Yizhou Wang, Xiangxi Shi, Yanzhi Wang, Pu Zhao, and Jiuxiang Gu. 2025a. Efficient reasoning with hidden thinking. Preprint, arXiv:2501.19201. · arXiv
  57. Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. 2025b. Codi: Compress- ing chain-of-thought into continuous space via self- distillation. Preprint, arXiv:2502.21074. · arXiv
  58. Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji- Rong Wen. 2025. R1-searcher: Incentivizing the search capability in llms via reinforcement learning. Preprint, arXiv:2503.05592. · arXiv
  59. DiJia Su, Hanlin Zhu, Yingchen Xu, Jiantao Jiao, Yuandong Tian, and Qinqing Zheng. 2025. To- ken assorted: Mixing latent and text tokens for improved language model reasoning. Preprint, arXiv:2502.03275. · arXiv
  60. Yutao Sun, Hangbo Bao, Wenhui Wang, Zhiliang Peng, Li Dong, Shaohan Huang, Jianyong Wang, and Furu Wei. 2024. Multimodal latent lan- guage modeling with next-token diffusion. Preprint, arXiv:2412.08635. · arXiv
  61. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Se- bastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and Jason Wei. 2023. Challenging BIG- bench tasks and whether chain-of-thought can solve them. In Findings of the Association for Computational Linguistics: ACL 2023, pages 13003- 13051, Toronto, Canada. Association for Computa- tional Linguistics. · DOI · Paper page
  62. Jihoon Tack, Jack Lanchantin, Jane Yu, Andrew Co- hen, Ilia Kulikov, Janice Lan, Shibo Hao, Yuan- dong Tian, Jason Weston, and Xian Li. 2025. Llm pretraining with continuous concepts. Preprint, arXiv:2502.08524. · arXiv
  63. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CommonsenseQA: A ques- tion answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4149-4158, Minneapolis, Minnesota. Asso- ciation for Computational Linguistics. · DOI
  64. Jiakai Tang, Sunhao Dai, Teng Shi, Jun Xu, Xu Chen, Wen Chen, Wu Jian, and Yuning Jiang. 2025. Think before recommend: Unleashing the latent reason- ing power for sequential recommendation. Preprint, arXiv:2503.22675. · arXiv
  65. Boshi Wang, Xiang Yue, Yu Su, and Huan Sun. 2024a. Grokked transformers are implicit reason- ers: A mechanistic journey to the edge of generaliza- tion. In Advances in Neural Information Processing Systems.
  66. Rui Wang, Hongru Wang, Boyang Xue, Jianhui Pang, Shudong Liu, Yi Chen, Jiahao Qiu, Derek Fai Wong, Heng Ji, and Kam-Fai Wong. 2025a. Har- nessing the reasoning economy: A survey of effi- cient reasoning for large language models. Preprint, arXiv:2503.24377. · arXiv
  67. Xinyi Wang, Lucas Caccia, Oleksiy Ostapenko, Xingdi Yuan, William Yang Wang, and Alessandro Sordoni. 2024b. Guiding language model reasoning with planning tokens. In First Conference on Language Modeling.
  68. Yiming Wang, Pei Zhang, Baosong Yang, Derek F. Wong, and Rui Wang. 2025b. Latent space chain-of- embedding enables output-free llm self-evaluation. In The Thirteenth International Conference on Learning Representations.
  69. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits rea- soning in large language models. Advances in neural information processing systems, 35:24824-24837.
  70. Bohong Wu, Shen Yan, Sijun Zhang, Jianqiao Lu, Yutao Zeng, Ya Wang, and Xun Zhou. 2025. Efficient pre- training length scaling. Preprint, arXiv:2504.14992. · arXiv
  71. Yige Xu, Xu Guo, Zhiwei Zeng, and Chunyan Miao. 2025a. Softcot: Soft chain-of-thought for efficient reasoning with llms. Preprint, arXiv:2502.12134. · arXiv
  72. Yige Xu, Xu Guo, Zhiwei Zeng, and Chunyan Miao. 2025b. Softcot: Soft chain-of-thought for efficient reasoning with llms. Preprint, arXiv:2502.12134. · arXiv
  73. Yige Xu, Xu Guo, Zhiwei Zeng, and Chunyan Miao. 2025c. Softcot++: Test-time scaling with soft chain- of-thought reasoning. Preprint, arXiv:2505.11484. · arXiv
  74. Hongfei Xue, Yufeng Tang, Hexin Liu, Jun Zhang, Xue- long Geng, and Lei Xie. 2025. Enhancing non-core language instruction-following in speech llms via semi-implicit cross-lingual cot reasoning. Preprint, arXiv:2504.20835. · arXiv
  75. Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, and Sebastian Riedel. 2024. Do large lan- guage models latently perform multi-hop reasoning? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10210-10229, Bangkok, Thailand. Association for Computational Linguistics. · DOI
  76. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christo- pher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answer- ing. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369-2380, Brussels, Belgium. Association for Computational Linguistics. · DOI · Paper page
  77. Jiacheng Ye, Shansan Gong, Liheng Chen, Lin Zheng, Jiahui Gao, Han Shi, Chuan Wu, Xin Jiang, Zhen- guo Li, Wei Bi, and Lingpeng Kong. 2024. Dif- fusion of thoughts: Chain-of-thought reasoning in diffusion language models. In Advances in Neural Information Processing Systems.
  78. Tian Ye, Zicheng Xu, Yuanzhi Li, and Zeyuan Allen- Zhu. 2025. Physics of language models: Part 2.1, grade-school math and the hidden reasoning pro- cess. In The Thirteenth International Conference on Learning Representations.
  79. Alexander Yom Din, Taelin Karidi, Leshem Choshen, and Mor Geva. 2024. Jump to conclusions: Short-cutting transformers with linear trans- formations. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 9615-9625, Torino, Italia. ELRA and ICCL.
  80. Ping Yu, Jing Xu, Jason Weston, and Ilia Kulikov. 2024. Distilling system 2 into system 1. Preprint, arXiv:2407.06023. · arXiv
  81. Qifan Yu, Zhenyu He, Sijie Li, Xun Zhou, Jun Zhang, Jingjing Xu, and Di He. 2025. Enhancing auto- regressive chain-of-thought through loop-aligned rea- soning. Preprint, arXiv:2502.08482. · arXiv
  82. Yijiong Yu. 2025. Do llms really think step-by-step in implicit reasoning? Preprint, arXiv:2411.15862. · arXiv
  83. Eric Zelikman, Georges Raif Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, and Noah Goodman. 2024. Quiet-STar: Language models can teach themselves to think before speaking. In First Conference on Language Modeling.
  84. Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. 2022. STar: Bootstrapping reasoning with reasoning. In Advances in Neural Information Processing Systems.
  85. Jason Zhang and Scott Viteri. 2025. Uncovering la- tent chain of thought vectors in language models. Preprint, arXiv:2409.14026. · arXiv
  86. Jintian Zhang, Yuqi Zhu, Mengshu Sun, Yujie Luo, Shuofei Qiao, Lun Du, Da Zheng, Huajun Chen, and Ningyu Zhang. 2025. Lightthinker: Thinking step- by-step compression. Preprint, arXiv:2502.15589. · arXiv
  87. Yang Zhang, Shixin Yang, Chenjia Bai, Fei Wu, Xiu Li, Zhen Wang, and Xuelong Li. 2024. Towards efficient llm grounding for embodied multi-agent collabora- tion. Preprint, arXiv:2405.14314. · arXiv
  88. Xueyang Zhou, Guiyao Tie, Guowen Zhang, Weidong Wang, Zhigang Zuo, Di Wu, Duanfeng Chu, Pan Zhou, Lichao Sun, and Neil Zhenqiang Gong. 2025. Large reasoning models in agent scenarios: Explor- ing the necessity of reasoning capabilities. Preprint, arXiv:2503.11074. · arXiv
  89. Yutao Zhu, Zhaoheng Huang, Zhicheng Dou, and Ji- Rong Wen. 2024. One token can help! learn- ing scalable and pluggable virtual tokens for retrieval-augmented large language models. Preprint, arXiv:2405.19670. · arXiv

Cited by

Sorted by citations

See all 1

Metadata

Authors
Chen, X., Zhao, A., Xia, H., Lu, X., Wang, H., Chen, Y., Zhang, W., Wang, J., Li, W., & Shen, X.
Affiliations
Hong Kong Polytechnic University (HK)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
Pages
1125-1143
DOI
10.48550/arxiv.2505.16782
arXiv
2505.16782
OpenAlex
W4416452771
Cited by
1
References
2
Citations by year
2026: 1
FWCI
8.93
Type
conference paper
Language
en
Open access
Licence
creativecommons.org/licenses/by-nc-sa/4.0/legalcode
Locations
  • Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
large language models, abstract thinking, mental representation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning. https://app.cochat.ai/papers/10.48550/arxiv.2505.16782
The paper
Chen, X., Zhao, A., Xia, H., Lu, X., Wang, H., Chen, Y., Zhang, W., Wang, J., Li, W., & Shen, X. (2025). Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning. arXiv. https://doi.org/10.48550/arxiv.2505.16782