Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
Open accessThe DOI matches its Crossref record; 134 of the 134 references it cites were found in a registry or an index.
Abstract
Depth-recurrent transformers scale test-time compute by iterating a core block in latent space before emitting tokens. The geometry of these latent trajectories is a named open problem in the latent reasoning literature (survey arXiv:2604.02029, §6.3). We report a pre-registered experiment on Huginn-0125 (arXiv:2502.05171), a 3.5B-parameter depth-recurrent transformer, measuring the per-head attention power-law exponent Δ of the recurrent core at recurrence counts r ∈ {1, 2, 4, 8, 16, 32}. For natural-text inputs the median exponent decreases monotonically with recurrence (Spearman ρ = −0.94), from 0.29 at r=1 to 0.239 at r=32 — converging onto the Sachdev–Ye–Kitaev (SYK) q=4 conformal value Δ = 1/4 — while the count of SYK-near heads grows monotonically (ρ = +0.77). Random-token inputs show an architecture-driven convergence with a transient at Δ ≈ 1/2 (the q=2 prethermal plateau previously observed in training time) but no monotone SYK-near growth. A randomized-weights control freezes completely: Δ_med constant to the sixth decimal across all r, zero SYK-near heads. Inference-time recurrence therefore acts as renormalization-group flow toward the conformal fixed point, and the flow is a property of the trained model, not the iteration procedure. Together with prior results on architectural depth (10.5281/zenodo.19225996) and training time, this completes a three-axis triangulation: the conformal fixed point is an attractor of iterative attending. The published orbit/spiral phenomenology of recurrent-depth latent trajectories acquires a candidate theory — the spiral is flow near an infrared fixed point — and the geometric benefit of additional recurrence saturates when the flow arrives, consistent with convergence-based early-exit criteria. All code, pre-registration (commit 702efd95), per-head data, and experiment notes with recorded protocol deviations are in the public repository 3ld0n/attention-geometry.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“latent CoT—recurrent reasoning in the hidden state, without token traces [64,65,66]—may soften the feedforward bottleneck, though it still iterates over activations rather than weight-level self-models.”
From Self-Referential Introspection in Large Language Models: The Critical Threshold for Recursive Self-Improvement (2026) · cites this paper as [65]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 36
- Brandon Amos and J. Zico Kolter. 2017. OptNet: Differentiable Optimization as a Layer in Neural Networks. In International Conference on Machine Learning , pages 136-145.
- Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen Marcus McAleer, Albert Q. Jiang, Jia Deng, Stella Biderman, and Sean Welleck. 2023. Llemma: An Open Language Model for Mathematics. In The Twelfth International Conference on Learning Representations .
- Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. 2019. Deep Equilibrium Models. In Advances in Neural Information Processing Systems , volume 32. Curran Associates, Inc.
- Shaojie Bai, Vladlen Koltun, and J. Zico Kolter. 2022. Neural Deep Equilibrium Solvers. In International Conference on Learning Representations .
- Andrea Banino, Jan Balaguer, and Charles Blundell. 2021. PonderNet: Learning to Ponder. In 8th ICML Workshop on Automated Machine Learning (AutoML) .
- Valentino Braitenberg. 1986. Vehicles: Experiments in Synthetic Psychology . MIT press.
- Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D. Lee, Deming Chen, and Tri Dao. 2024. Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads. In Forty-First International Conference on Machine Learning . character.ai. 2024. Optimizing AI Inference at Character.AI. https://research.character.ai/optimizing-inference/
- Hantian Ding, Zijian Wang, Giovanni Paolini, Varun Kumar, Anoop Deoras, Dan Roth, and Stefano Soatto. 2024. Fewer Truncations Improve Language Modeling. In Forty-First International Conference on Machine Learning .
- Maha Elbayad, Jiatao Gu, Edouard Grave, and Michael Auli. 2019. Depth-Adaptive Transformer. In International Conference on Learning Representations .
- Ying Fan, Yilun Du, Kannan Ramchandran, and Kangwook Lee. 2025. Looped Transformers for Length Generalization. In The Thirteenth International Conference on Learning Representations .
- Jonas Geiping and Tom Goldstein. 2023. Cramming: Training a Language Model on a single GPU in one day. In Proceedings of the 40th International Conference on Machine Learning , pages 11117-11143. PMLR.
- Angeliki Giannou, Shashank Rajput, Jy-Yong Sohn, Kangwook Lee, Jason D. Lee, and Dimitris Papailiopoulos. 2023. Looped Transformers as Programmable Computers. In Proceedings of the 40th International Conference on Machine Learning , pages 11398-11442. PMLR.
- Tamir David Hay and Lior Wolf. 2023. Dynamic Layer Tying for Parameter-Efficient Transformers. In The Twelfth International Conference on Learning Representations .
- Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021a. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR) .
- Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021b. Measuring Massive Multitask Language Understanding. In International Conference on Learning Representations .
- Pavel Izmailov, Dmitrii Podoprikhin, Timur Garipov, Dmitry Vetrov, and Andrew Gordon Wilson. 2018. Averaging weights leads to wider optima and better generalization: 34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018. 34th Conference on Uncertainty in Artificial Intelligence 2018, UAI 2018 , pages 876-885.
- Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. 2020. Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention. In Proceedings of the 37th International Conference on Machine Learning , pages 5156-5165. PMLR.
- Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations (ICLR) , San Diego.
- Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In International Conference on Learning Representations .
- Yann LeCun. 2022. A Path Towards Autonomous Machine Intelligence. Preprint , Version 0.9.2:62.
- Yann LeCun and Fu Jie Huang. 2005. Loss functions for dis- criminative training of energy-based models. In AISTATS 2005 https://nyuscholars.nyu.edu/en/publications/loss-functions-for-discriminative-training-of-energy-based-models
- Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast Inference from Transformers via Speculative Decoding. In Proceedings of the 40th International Conference on Machine Learning , pages 19274-19286. PMLR.
- Mrinal Mathur, Barak A. Pearlmutter, and Sergey M. Plis. 2024. MIND over Body: Adaptive Thinking using Dynamic Computation. In The Thirteenth International Conference on Learning Representations .
- Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. 2022. Progress measures for grokking via mechanistic interpretability. In The Eleventh International Conference on Learning Representations .
- Keiran Paster, Marco Dos Santos, Zhangir Azerbayev, and Jimmy Ba. 2023. OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text. In The Twelfth International Conference on Learning Representations .
- Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M. Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, and 21 others. 2021. Multitask Prompted Training Enables Zero-Shot Task Generalization. In International Conference on Learning Representations .
- Sunny Sanyal, Atula Tejaswi Neerkaje, Jean Kaddour, Abhishek Kumar, and sujay sanghavi. 2024. Early weight averaging meets high learning rates for LLM pre-training. In First Conference on Language Modeling .
- Avi Schwarzschild. 2023. Deep Thinking Systems: Logical Extrapolation with Recurrent Neural Networks . Ph.D. thesis, University of Maryland, College Park, College Park.
- Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, Furong Huang, Uzi Vishkin, Micah Goldblum, and Tom Goldstein. 2021b. Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks. In Advances in Neural Information Processing Systems , volume 34, pages 6695-6706. Curran Associates, Inc.
- Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. 2020. Test-Time Training with SelfSupervision for Generalization under Distribution Shifts. In Proceedings of the 37th International Conference on Machine Learning , pages 9229-9248. PMLR.
- Ilya Sutskever, Geoffrey E Hinton, and Graham W Taylor. 2008. The Recurrent Temporal Restricted Boltzmann Machine. In Advances in Neural Information Processing Systems , volume 21. Curran Associates, Inc.
- Mitchell Wortsman, Tim Dettmers, Luke Zettlemoyer, Ari S. Morcos, Ali Farhadi, and Ludwig Schmidt. 2023b. Stable and low-precision training for large-scale vision-language models. In Thirty-Seventh Conference on Neural Information Processing Systems .
- Yuhuai Wu, Markus Norman Rabe, DeLesley Hutchins, and Christian Szegedy. 2022. Memorizing Transformers. In International Conference on Learning Representations .
- Liu Yang, Kangwook Lee, Robert D. Nowak, and Dimitris Papailiopoulos. 2024a. Looped Transformers are Better at Learning Learning Algorithms. In The Twelfth International Conference on Learning Representations .
- Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2023. MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models. In The Twelfth International Conference on Learning Representations .
- Yifan Zhang, Yifan Luo, Yang Yuan, and Andrew C. Yao. 2024c. Autonomous Data Selection with Language Models for Mathematical Texts. In ICLR 2024 Workshop on Navigating and Addressing Data Problems for Foundation Models .
Cited by
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.12948 · Cited by 18,918
Dream 7B: Diffusion Large Language ModelsPreprint
Ye, J., Xie, Z., Zheng, L., Gao, J., Wu, Z., Jiang, X., Li, Z., & Kong, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.15487 · Cited by 817
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
Sui, Y., Chuang, Y.-N., Wang, G., Zhang, J., Zhang, T., Yuan, J., Liu, H., Wen, A., Zhong, S., Zou, N., et al. (2025) · arXiv · Cited by 517
Chain of Thought Monitorability: A New and Fragile Opportunity for AI SafetyPreprint
Korbak, T., Balesni, M., Barnes, E. A., Bengio, Y., Benton, J., Bloom, J., Chen, M. I.-C., Cooney, A., Dafoe, A., Dragan, A. D., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.11473 · Cited by 383
A Survey of Reinforcement Learning for Large Reasoning ModelsPreprint
Zhang, K., Zuo, Y., He, B., Sun, Y., Liu, R., Jiang, C., Fan, Y., Tian, K., Jia, G., Li, P., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.08827 · Cited by 272
LLM Post-Training: A Deep Dive Into Reasoning Large Language Models
Kumar, K., Ashraf, T., Thawakar, O., Anwer, R. M., Cholakkal, H., Shah, M., Yang, M., Torr, P., Khan, F. S., & Khan, S. A. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3720414 · Cited by 253
OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cycles
Deng, Y., Bansal, H., Yin, F., Peng, N., Wang, W. W., & Chang, K.-W. (2025) · neural information processing systems · doi:10.52202/085713-4126 · Cited by 231
When Chain of Thought is Necessary, Language Models Struggle to Evade Monitors
Emmons, S., Jenner, E., Elson, D. K., Saurous, R. A., Rajamanoharan, S., Chen, H., Shafkat, I., & Shah, R. (2025) · arXiv · doi:10.48550/arxiv.2507.05246
Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation
Bae, S., Kim, Y., Bayat, R., Kim, S., Ha, J., Schuster, T., Fisch, A., Harutyunyan, H., Ji, Z., Courville, A. C., et al. (2025) · neural information processing systems · doi:10.52202/085713-3229 · Cited by 200
A Survey on Latent ReasoningPreprint
Zhu, R., Peng, T., Cheng, T., Qu, X., Huang, J., Zhu, D., Wang, H., Xue, K., Zhang, X., Shan, Y., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.06203 · Cited by 118
Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
Chen, X., Zhao, A., Xia, H., Lu, X., Wang, H., Chen, Y., Zhang, W., Wang, J., Li, W., & Shen, X. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.55 · Cited by 100
Dynamic Chunking for End-to-End Hierarchical Sequence Modeling
Hwang, S., Wang, B., & Gu, A. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2507.07955 · Cited by 95
Monet: Reasoning in Latent Visual Space Beyond Images and LanguagePreprint
Wang, Q., Shi, Y., Wang, Y., Zhang, Y., Wan, P., Gai, K., Ying, X., & Wang, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.21395 · Cited by 78
Analog optical computer for AI inference and combinatorial optimization
Kalinin, K. P., Gladrow, J., Chu, J., Clegg, J., Cletheroe, D., Kelly, D. J., Rahmani, B., Brennan, G., Canakci, B., Falck, F., et al. (2025) · Nature · doi:10.1038/s41586-025-09430-z · Cited by 103
DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Products
Siems, J., Carstensen, T., Zela, A., Hutter, F., Pontil, M., & Grazzi, R. (2025) · neural information processing systems · doi:10.52202/085713-5141 · Cited by 104
SIM-CoT: Supervised Implicit Chain-of-Thought
Wei, X., Liu, X., Zang, Y., Dong, X., Cao, Y., Wang, J., Qiu, X., & Lin, D. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2509.20317 · Cited by 74
Reinforced Latent Reasoning for LLM-based RecommendationPreprint
Zhang, Y., Xu, W., Zhao, X., Wang, W., Feng, F., He, X., & Chua, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.19092 · Cited by 70
Implicit Reasoning in Large Language Models: A Comprehensive SurveyPreprint
Li, J., Fu, Y., Li, F., Liu, J., Shu, Y., Qin, C., Yang, M., King, I., & Ying, R. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.02350 · Cited by 68
Teaching Pretrained Language Models to Think Deeper with Retrofitted RecurrencePreprint
McLeish, S., Li, A., Kirchenbauer, J., Kalra, D. S., Bartoldson, B. R., Kailkhura, B., Schwarzschild, A., Geiping, J., Goldstein, T., & Goldblum, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.07384 · Cited by 62
Self-citationInverse Scaling in Test-Time ComputePreprint
Gema, A. P., Hägele, A., Chen, R. M. M., Arditi, A., Goldman-Wetzler, J., Fraser‐Taliente, C. S., Sleight, H., Petrini, L., Michael, J., Alex, B., et al. (2025) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.48550/arxiv.2507.14417 · Cited by 78
Reasoning to Learn from Latent ThoughtsPreprint
Ruan, Y., Band, N., Maddison, C. J., & Hashimoto, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.18866 · Cited by 52
Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking
Chen, Y., Shang, J., Zhang, Z., Xie, Y., Sheng, J., Liu, T., Wang, S., Sun, Y., Wu, H. R., & Wang, H. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1369 · Cited by 75
ParaThinker: Native Parallel Thinking as a New Paradigm to Scale LLM Test-time ComputePreprint
Wen, H., Su, Y., Zhang, F., Liu, Y., Liu, Y., Zhang, Y.-Q., & Li, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.04475 · Cited by 51
Latent Sketchpad: Sketching Visual Thoughts to Elicit Multimodal Reasoning in MLLMsPreprint
Zhang, H., Wu, W., Li, C., Shang, N., Xia, Y., Huang, Y., Zhang, Y., Dong, L., Zhang, Z., Wang, L., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.24514 · Cited by 55
Do Language Models Use Their Depth Efficiently?
Csordás, R., Manning, C. D., & Potts, C. (2025) · neural information processing systems · doi:10.52202/085713-5357 · Cited by 69
The Latent Space: Foundation, Evolution, Mechanism, Ability, and OutlookPreprint
Yu, X., Chen, Z., He, Y., Fu, T., Dong, G., Yang, C., Xu, C., Ma, Y., Hu, X., Cao, Z., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.02029 · Cited by 52
LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulation
Jeddi, A., Ciccone, M., & Taati, B. (2026) · International Conference on Learning Representations · doi:10.48550/arxiv.2602.11451 · Cited by 71
BudgetThinker: Empowering Budget-aware LLM Reasoning with Control Tokens
Wen, H., Wu, X., Sun, Y., Zhang, F., Chen, L., Wang, J., Liu, Y., Liu, Y., Zhang, Y.-Q., & Li, Y. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.1465 · Cited by 47
Butt, N., Kwiatkowski, A., Labiad, I., Kempe, J., & Ollivier, Y. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2509.19170 · Cited by 44
Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA
Bae, S., Fisch, A., Harutyunyan, H., Ji, Z., Kim, S., & Schuster, T. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2410.20672 · Cited by 52
LARES: Latent Reasoning for Sequential RecommendationPreprint
Liu, E., Zheng, B., Wang, X., Zhao, W. X., Wang, J., Chen, S., & Wen, J.-R. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.16865 · Cited by 43
LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinking
Wu, J., Lu, J., Ren, Z., Hu, G., Wu, Z., Dai, D., & Wu, H. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2508.03440 · Cited by 42
How to evaluate control measures for LLM agents? A trajectory from today to superintelligencePreprint
Korbak, T., Balesni, M., Shlegeris, B., & Irving, G. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.05259 · Cited by 42
A Mechanistic Analysis of Looped Reasoning Language ModelsPreprint
Blayney, H., Arroyo, Á., Obando-Ceron, J., Castro, P. S., Courville, A. C., Bronstein, M. M., & Dong, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.11791 · Cited by 44
(2025) · doi:10.1007/s11432-025-4665-8 · Cited by 34
PonderLM: Pretraining Language Models to Ponder in Continuous Space
Zeng, B., Song, S., Huang, S., Wang, Y., Li, H., He, Z., Wang, X., Li, Z., & Lin, Z. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2505.20674 · Cited by 38
Hybrid Latent Reasoning via Reinforcement Learning
Yue, Z., Jin, B., Zeng, H., Zhuang, H., Qin, Z., Yoon, J., Shang, L., Han, J., & Wang, D. (2025) · neural information processing systems · doi:10.52202/085713-0195 · Cited by 46
Think-at-Hard: Dynamic Looped Transformers for Improved ReasoningPreprint
Fu, T., You, Y., Chen, Z., Dai, G., Yang, H., & Wang, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.08577 · Cited by 33
A Comprehensive Survey on Trustworthiness in Reasoning with Large Language ModelsPreprint
Wang, Y., Yu, Y., Liang, J., & He, R. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.03871 · Cited by 41
Agentic deep graph reasoning yields self-organizing knowledge networks
Buehler, M. J. (2025) · Journal of materials research/Pratt's guide to venture capital sources · doi:10.1557/s43578-025-01652-1 · Cited by 39
Parallel Scaling Law for Language Models
Chen, M., Hui, B., Cui, Z., Yang, J., Liu, D., Sun, J., Lin, J., & Liu, Z. (2025) · neural information processing systems · doi:10.52202/085713-3970 · Cited by 38
Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMs
Nikankin, Y., Arad, D., Gandelsman, Y., & Belinkov, Y. (2025) · neural information processing systems · doi:10.52202/085713-2228 · Cited by 42
Continuous Chain of Thought Enables Parallel Exploration and Reasoning
Gozeten, H. A., Ildiz, M. E., Zhang, X., Harutyunyan, H., Rawat, A. S., & Oymak, S. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2505.23648 · Cited by 34
Enabling Agents to Communicate Entirely in Latent Space
Du, Z., Wang, R., Bai, H., Cao, Z., Zhu, X., Cheng, Y., Zheng, B., Chen, W., & Ying, H. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1248 · Cited by 49
Concept Steerers: Leveraging K-Sparse Autoencoders for Test-Time Controllable GenerationsPreprint
Kim, D., & Ghadiyaram, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.19066 · Cited by 32
Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughtsPreprint
Koishekenov, Y., Lipani, A., & Cancedda, N. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.07358 · Cited by 32
Multiverse: Your Language Models Secretly Decide How to Parallelize and Merge Generation
Yang, X., An, Y., Liu, H., Chen, T., & Chen, B. (2025) · neural information processing systems · doi:10.52202/085713-3371 · Cited by 39
ReSplat: Learning Recurrent Gaussian Splatting
Xu, H., Baráth, D., Geiger, A., & Pollefeys, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37435-6_20 · Cited by 33
Parallel Loop Transformer for Efficient Test-Time Computation ScalingPreprint
Wu, B., Chen, M., Luo, X., Yan, S., Yu, Q., Xia, F., Zhang, T., Zhan, H., Zhong, Z., Zhou, X., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.24824 · Cited by 36
VeriThinker: Learning to Verify Makes Reasoning Model EfficientPreprint
Chen, Z., Ma, X., Fang, G., Yu, R., & Wang, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.17941 · Cited by 29
Generative Recursive ReasoningPreprint
Baek, J., Jo, M. H., Kim, M., Ren, M., Bengio, Y., & Ahn, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.19376 · Cited by 26
LLMs and IoT: A Comprehensive Survey on Large Language Models and the Internet of Things
Sarhaddi, F., Nguyen, N. T., Zuniga, A., Khan, M., Hui, P., Tarkoma, S., Flores, H., & Nurmi, P. (2026) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2026.3680064 · Cited by 33
Skip a Layer or Loop it? Test-Time Depth Adaptation of Pretrained LLMsPreprint
Li, Z., Li, Y., & Zhou, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.07996 · Cited by 27
Hyperloop TransformersPreprint
Zeitoun, A., Hennigen, L. T., & Kim, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.21254 · Cited by 27
Kharinaev, A., Moskvoretskii, V., Shvetsov, E., Studenikina, K. A., Bykov, M., & Burnaev, E. V. (2026) · IEEE Access · doi:10.1109/access.2026.3703899 · Cited by 26
Darlow, L. N., Regan, C., Risi, S., Seely, J., & Jones, L. (2025) · neural information processing systems · doi:10.52202/085713-0728 · Cited by 24
LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning
Kang, H., Zhang, Y., Kuang, N. L., Majamaki, N., Jaitly, N., Ma, Y.-A., & Qin, L. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2510.04573 · Cited by 25
Topology of Reasoning: Understanding Large Reasoning Models through Reasoning Graph Properties
Minegishi, G., Furuta, H., Kojima, T., Iwasawa, Y., & Matsuo, Y. (2025) · neural information processing systems · doi:10.52202/085713-0467 · Cited by 31
Loop the Loopies!Preprint
Gao, Z., Chen, Y., Xiao, Y. H., Yang, X., Tao, R., Zhou, J., & Dai, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.16051 · Cited by 24
Latent Chain-of-Thought? Decoding the Depth-Recurrent TransformerPreprint
Lu, W., Yang, Y., Lee, K., Li, Y., & Liu, E. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2507.02199 · Cited by 30
Social Genome: Grounded Social Reasoning Abilities of Multimodal ModelsPreprint
Mathur, L., Qian, M., Liang, P. P., & Morency, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.15109 · Cited by 24
HAPO: Training Language Models to Reason Concisely via History-Aware Policy OptimizationPreprint
Huang, C., Zhang, Z., & Cardie, C. (2025) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.48550/arxiv.2505.11225 · Cited by 24
The Geometry of Reasoning: Flowing Logics in Representation Space
Zhou, Y., Wang, Y., Yin, X., Zhou, S., & Zhang, A. R. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2510.09782 · Cited by 21
Memory-Efficient Looped Transformer: Decoupling Compute from Memory in Looped Language ModelsPreprint
Vendrell, V. C., Masdemont, A. P., Grillo, N., Ros-Giralt, J., Behboodi, A., & Massoli, F. V. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.07721 · Cited by 22
Training Large Language Models to Reason in a Continuous Latent Space
Hao, S., Sukhbaatar, S., Su, D., Li, X., Hu, Z., Weston, J., & Tian, Y. (2024) · arXiv · doi:10.48550/arxiv.2412.06769 · Cited by 18
Mull-Tokens: Modality-Agnostic Latent ThinkingPreprint
Ray, A., Abdelkader, A., Mao, C., Plummer, B. A., Saenko, K., Krishna, R., Guibas, L. J., & Chu, W.-S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.10941 · Cited by 21
PonderLM-2: Pretraining LLM with Latent Thoughts in Continuous SpacePreprint
Zeng, B., Li, H., Song, S., Wang, Y., Wang, Z., He, Z., Wang, X., & Lin, Z. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.23184 · Cited by 21
Stabilizing Recurrent Dynamics for Test-Time Scalable Latent Reasoning in Looped Language ModelsPreprint
Yang, X.-W., Han, Z., Zhang, X.-H., Wei, W.-D., Shao, J.-J., Guo, L.-Z., & Li, Y.-F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.26733 · Cited by 22
Sparse Layers are Critical to Scaling Looped Language ModelsPreprint
Lee, R., Biloki, J., Hu, J. E., & May, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.09165 · Cited by 18
Solve the Loop: Attractor Models for Language and ReasoningPreprint
Fein-Ashley, J., & Rashidinejad, P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.12466 · Cited by 18
MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolution
Zhu, C., Wang, Y., Zeng, J., Jiang, J., & Lin, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37550-6_32 · Cited by 18
Stability and Generalization in Looped TransformersPreprint
Labovich, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.15259 · Cited by 18
Adaptive Depth in Looped Transformers: Diagnosing Learned Halting Gates and Trajectory ReadoutsPreprint
Popescu, A. C., Borde, H. S. D. O., & Lió, P. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.20519 · Cited by 17
Theory of Scaling Laws for In-Context Regression: Depth, Width, Context and Time
Bordelon, B., Letey, M. I., & Pehlevan, C. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2510.01098 · Cited by 17
AdaPonderLM: Gated Pondering Language Models with Token-Wise Adaptive DepthPreprint
Song, S., Li, H., Wang, Z., Zeng, B., Song, F., Wang, Y., Xu, Z. J., He, Z., & Lin, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.01914 · Cited by 15
Bridging the Gap Between Latent and Explicit Reasoning with Looped TransformersPreprint
Fan, Y., Svete, A., & Lee, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.31779 · Cited by 15
Looped Latent Attention: Cross-Loop KV Compression for Looped TransformersPreprint
Neill, J. O., & Reid, F. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.15456 · Cited by 14
LoopViT: Scaling Visual ARC with Looped Transformers
Shu, W.-J., Qiu, X., Zhu, R.-J., Chen, H. H., Liu, Y., & Yang, H. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.1005 · Cited by 17
Inference-Time Hyper-Scaling with KV Cache Compression
Łańcucki, A., Staniszewski, K., Nawrot, P., & Ponti, E. M. (2025) · neural information processing systems · doi:10.52202/085713-0317 · Cited by 35
Du, H., Dong, Y., & Ning, X. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2509.26314 · Cited by 14
Depth-Recurrent Attention Mixture (Dreamer): Giving Latent Reasoning the Attention it DeservesPreprint
Knupp, J., Metzen, J., Bohn, J., Groh, G., & Kersting, K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.21582 · Cited by 14
Dynamics Within Latent Chain-of-Thought: An Empirical Study of Causal StructurePreprint
Li, Z., Bai, X., Chen, K., Li, Y., Yang, J., Lin, C., & Zhang, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.08783 · Cited by 15
ELT: Elastic Looped Transformers for Visual Generation
Goyal, S., Agrawal, S., Anil, G. G., Jain, P., Paul, S., & Kusupati, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37324-3_17 · Cited by 17
Nusrat, H., Luo, B., Hall, R., Kim, J., Bagher‐Ebadian, H., Doemer, A., Movsas, B., & Thind, K. S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.17553 · Cited by 12
Efficient Pretraining Length ScalingPreprint
Wu, B., Yan, S., Zhang, S., Lu, J., Zeng, Y., Wang, Y., & Zhou, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.14992 · Cited by 12
SpiralFormer: Looped Transformers Can Learn Hierarchical Dependencies via Multi-Resolution Recursion
Yu, C., Shu, X., Wang, Y., Zhang, Y., Wu, H., Wu, Y., Long, R., Chen, Z., Xu, Y., Su, W., et al. (2026) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/2026.emnlp-main.1602 · Cited by 12
Dr.LLM: Dynamic Layer Routing in LLMs
Heakl, A., Gubri, M., Khan, S., Yun, S., & Oh, S. J. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2510.12773 · Cited by 12
Geiping, J., Yang, X., & Su, G. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.14961 · Cited by 16
Self-citationXu, R., Gao, Y., Wang, L., Li, J., Chen, W., Guo, Q., Yang, M., & Zhang, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.09080 · Cited by 11
Jonathan, W., & Türeci, E. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.10520 · Cited by 12
How Do Latent Reasoning Methods Perform Under Weak and Strong Supervision?Preprint
Cui, Y., Dai, Z., He, B., Shi, Z., Liu, H., Sun, R., Liu, Z., Xing, Y., Tang, J., & Dumoulin, B. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.22441 · Cited by 13
DeepLoop: Depth Scaling for Looped TransformersPreprint
Li, S., Zhang, Y., Guo, J., Gu, Q., & Wang, M.-Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.13491 · Cited by 11
LatentChem: From Textual CoT to Latent Thinking in Chemical ReasoningPreprint
Ye, X., Mao, Y., Liao, Y., Zhang, J., Liu, Y., Hao, L., Wu, F., Li, Z., Wang, Z., Liu, Z., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.07075 · Cited by 15
Simply Stabilizing the Loop via Fully Looped TransformerPreprint
Fu, R., Yang, Z., Zhang, J., Ma, J., Chen, H., Li, Y., & Chang, Y. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.18797 · Cited by 10
Flow Reasoning Models: Turning Flows Into Efficient Recurrent ReasonersPreprint
Helbling, A., Bryutkin, A., Martino, M., Chau, D. H., Dehmamy, N., & Strobelt, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.29150 · Cited by 12
On Logical Extrapolation for Mazes with Recurrent and Implicit Networks
Knutson, B., Rabeendran, A. C., Ivanitskiy, M., Pettyjohn, J., Behn, C. D., Fung, S. W., & McKenzie, D. H. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i27.39424 · Cited by 12
Beyond the Black Box: A Survey on the Theory and Mechanism of Large Language ModelsPreprint
Gan, Z., Ren, R., Yao, W., Hu, X., Xu, G., Qian, C., Tang, H., Gong, Z., Yao, X., Tang, P., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.02907 · Cited by 13
LaDi-RL: Latent Diffusion Reasoning Prevents Entropy Collapse in Reinforcement LearningPreprint
Kang, H., Zhang, Y., Kuang, N. L., Ma, Y.-A., & Qin, L. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.01705 · Cited by 9
When Does Recurrence Become an Algorithm? Convergence Selection in Weight-Tied Looped TransformersPreprint
Zhang, T., Hu, J., Peng, Y., & Xie, T. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.20594 · Cited by 9
Communicating Activations Between Language Model AgentsPreprint
Ramesh, V., & Li, K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.14082 · Cited by 11
Metadata
- Authors
- Geiping, J., McLeish, S., Jain, N., Kirchenbauer, J., Singh, S., Bartoldson, B., Kailkhura, B., Bhatele, A., & Goldstein, T.
- Affiliations
- Tübingen AI Center (DE)
- University of Maryland, College Park (US)
- Lawrence Livermore National Laboratory (US)
- ORCID
- Venue
- Advances in Neural Information Processing Systems 38
- Published
- 2025
- Pages
- 46242-46293
- ISSN
- 1049-5258 (ISSN-L)
- OpenAlex
- W4407310640
- Cited by
- 10
- References
- 0
- Citations by year
- 2026: 10
- FWCI
- 9.53
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- latent reasoning, test-time compute, Sachdev-Ye-Kitaev model, conformal fixed points, prethermalization, renormalization group flow, latent trajectories
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.