CoChat
Sign Up
PreprintCited by 1

The DOI matches its DataCite record.

Less is More: Recursive Reasoning with Tiny Networks

Jolicoeur-Martineau, Alexia·2025·arXiv
DOI 10.48550/arxiv.2510.04871arXiv 2510.04871
Open this paper in CoChat — free account

Abstract

Hierarchical Reasoning Model (HRM) is a novel approach using two small neural networks recursing at different frequencies. This biologically inspired method beats Large Language models (LLMs) on hard puzzle tasks such as Sudoku, Maze, and ARC-AGI while trained with small models (27M parameters) on small data (around 1000 examples). HRM holds great promise for solving hard problems with small networks, but it is not yet well understood and may be suboptimal. We propose Tiny Recursive Model (TRM), a much simpler recursive reasoning approach that achieves significantly higher generalization than HRM, while using a single tiny network with only 2 layers. With only 7M parameters, TRM obtains 45% test-accuracy on ARC-AGI-1 and 8% on ARC-AGI-2, higher than most LLMs (e.g., Deepseek R1, o3-mini, Gemini 2.5 Pro) with less than 0.01% of the parameters.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2510.04871

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 31

  1. Bai, S., Kolter, J. Z., and Koltun, V. Deep equilibrium models. Advances in neural information processing systems, 32, 2019.
  2. Bai, X. and Melas-Kyriazi, L. Fixed point diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9430- 9440, 2024.
  3. Brock, A., Donahue, J., and Simonyan, K. Large scale gan training for high fidelity natural image synthe- sis. arXiv preprint arXiv:1809.11096, 2018. · arXiv
  4. Chollet, F. On the measure of intelligence. arXiv preprint arXiv:1911.01547, 2019. · arXiv
  5. Chollet, F., Knoop, M., Kamradt, G., Landers, B., and Pinkard, H. Arc-agi-2: A new challenge for frontier ai reasoning systems. arXiv preprint arXiv:2505.11831, 2025. · arXiv
  6. Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. Palm: Scaling lan- guage modeling with pathways. Journal of Machine Learning Research, 24(240):1-113, 2023.
  7. Elman, J. L. Finding structure in time. Cognitive science, 14(2):179-211, 1990.
  8. Fedus, W., Zoph, B., and Shazeer, N. Switch transform- ers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Re- search, 23(120):1-39, 2022.
  9. Geng, Z. and Kolter, J. Z. Torchdeq: A library for deep equilibrium models. arXiv preprint arXiv:2310.18605, 2023. · arXiv
  10. Hendrycks, D. and Gimpel, K. Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415, 2016. · arXiv
  11. Jang, Y., Kim, D., and Ahn, S. Hierarchical graph generation with k2-trees. In ICML 2023 Workshop on Structured Probabilistic Inference Generative Modeling, 2023.
  12. Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020. · arXiv
  13. Kingma, D. P. and Ba, J. Adam: A method for stochas- tic optimization. arXiv preprint arXiv:1412.6980, 2014. · arXiv
  14. Krantz, S. G. and Parks, H. R. The implicit function theorem: history, theory, and applications. Springer Science & Business Media, 2002.
  15. LeCun, Y. Une procedure d'apprentissage ponr reseau a seuil asymetrique. Proceedings of cognitiva 85, pp. 599-604, 1985.
  16. Lehnert, L., Sukhbaatar, S., Su, D., Zheng, Q., Mcvay, P., Rabbat, M., and Tian, Y. Beyond a*: Better planning with transformers via search dynamics bootstrap- ping. arXiv preprint arXiv:2402.14083, 2024. · arXiv
  17. Loshchilov, I. and Hutter, F. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. · arXiv
  18. Moskvichev, A., Odouard, V. V., and Mitchell, M. The conceptarc benchmark: Evaluating understanding and generalization in the arc domain. arXiv preprint arXiv:2305.07141, 2023. · arXiv
  19. Palm, R., Paquet, U., and Winther, O. Recurrent re- lational networks. Advances in neural information processing systems, 31, 2018.
  20. Prieto, L., Barsbey, M., Mediano, P. A., and Birdal, T. Grokking at the edge of numerical stability. arXiv preprint arXiv:2501.04697, 2025. · arXiv
  21. Shazeer, N. Glu variants improve transformer. arXiv preprint arXiv:2002.05202, 2020. · arXiv
  22. Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., and Dean, J. Outrageously large neu- ral networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538, 2017. · arXiv
  23. Snell, C., Lee, J., Xu, K., and Kumar, A. Scaling llm test-time compute optimally can be more effec- tive than scaling model parameters. arXiv preprint arXiv:2408.03314, 2024. · arXiv
  24. Song, Y. and Ermon, S. Improved techniques for train- ing score-based generative models. Advances in neural information processing systems, 33:12438-12448, 2020.
  25. Tolstikhin, I. O., Houlsby, N., Kolesnikov, A., Beyer, L., Zhai, X., Unterthiner, T., Yung, J., Steiner, A., Keysers, D., Uszkoreit, J., et al. Mlp-mixer: An all-mlp architecture for vision. Advances in neural information processing systems, 34:24261-24272, 2021.
  26. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  27. Wang, G., Li, J., Sun, Y., Chen, X., Liu, C., Wu, Y., Lu, M., Song, S., and Yadkori, Y. A. Hierarchical reasoning model. arXiv preprint arXiv:2506.21734, 2025. · arXiv
  28. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of-thought prompting elicits reasoning in large language mod- els. Advances in neural information processing systems, 35:24824-24837, 2022.
  29. Werbos, P. Beyond regression: New tools for predic- tion and analysis in the behavioral sciences. PhD thesis, Committee on Applied Mathematics, Harvard University, Cambridge, MA, 1974.
  30. Werbos, P. J. Generalization of backpropagation with application to a recurrent gas market model. Neural networks, 1(4):339-356, 1988.
  31. Zhang, B. and Sennrich, R. Root mean square layer normalization. Advances in Neural Information Pro- cessing Systems, 32, 2019.

Cited by

Sorted by citations

Metadata

Authors
Jolicoeur-Martineau, A.
Venue
arXiv
Publisher
arXiv
Published
2025
DOI
10.48550/arxiv.2510.04871
arXiv
2510.04871
References
0
Type
Preprint
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review