NumeroLogic: Number Encoding for Enhanced LLMs’ Numerical Reasoning
Open accessAbstract
Language models struggle with handling numerical data and performing arithmetic operations.We hypothesize that this limitation can be partially attributed to non-intuitive textual numbers representation.When a digit is read or generated by a causal language model it does not know its place value (e.g.thousands vs. hundreds) until the entire number is processed.To address this issue, we propose a simple adjustment to how numbers are represented by including the count of digits before each number.For instance, instead of "42", we suggest using "2:42" as the new format.This approach, which we term NumeroLogic, offers an added advantage in number generation by serving as a Chain of Thought (CoT).By requiring the model to consider the number of digits first, it enhances the reasoning process before generating the actual number.We use arithmetic tasks to demonstrate the effectiveness of the NumeroLogic formatting.We further demonstrate NumeroLogic applicability to general natural language modeling, improving language understanding performance in the MMLU benchmark.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLMs struggle to produce reliable prediction probabilities.
“However, they struggle to produce reliable prediction probabilities, which are crucial for transparency and decision-making.”
From Probabilistic medical predictions of large language models (2024) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 10
- James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jian- feng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. 2023. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8.
- Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebas- tian Gehrmann, et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learn- ing Research, 24(240):1-113.
- Philip Gage. 1994. A new algorithm for data compres- sion. The C Users Journal, 12(2):23-38.
- Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2021a. Aligning ai with shared human values. Pro- ceedings of the International Conference on Learning Representations (ICLR).
- Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Stein- hardt. 2021b. Measuring massive multitask language understanding. Proceedings of the International Con- ference on Learning Representations (ICLR).
- Jeonghwan Kim, Giwon Hong, Kyung-min Kim, Junmo Kang, and Sung-Hyon Myaeng. 2021. Have you seen that number? investigating extrapolation in question answering models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7031-7037, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. · DOI
- Nayoung Lee, Kartik Sreenivasan, Jason D. Lee, Kang- wook Lee, and Dimitris Papailiopoulos. 2024. Teach- ing arithmetic to small transformers. In The Twelfth International Conference on Learning Representa- tions.
- Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909. · arXiv
- Alex Warstadt, Aaron Mueller, Leshem Choshen, Ethan Wilcox, Chengxu Zhuang, Juan Ciro, Rafael Mos- quera, Bhargavi Paranjabe, Adina Williams, Tal Linzen, et al. 2023. Findings of the babylm chal- lenge: Sample-efficient pretraining on developmen- tally plausible corpora. In Proceedings of the BabyLM Challenge at the 27th Conference on Com- putational Natural Language Learning.
- Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits rea- soning in large language models. Advances in neural information processing systems, 35:24824-24837.
Cited by
Sorted by citations
Probabilistic medical predictions of large language models
Gu, B., Desai, R. J., Lin, K. J., & Yang, J. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01366-4 · Cited by 50 · see citing works
Survey on Tabular Data Privacy and Synthetic Data Generation in Industry 4.0
Koubeissy, H., Amine, A., Kamradt, M., & Makhoul, A. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06823-5 · Cited by 8 · see citing works
Wang, Y., Meng, X., Wang, Y., Liang, J., Wei, J., Zhang, H., & Zhao, D. (2024) · Qeios · doi:10.32388/1smziq · Cited by 5 · see citing works
Improving RAG for Personalization with Author Features and Contrastive Examples
Yazan, M., Verberne, S., & Situmeang, F. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88714-7_40 · Cited by 4 · see citing works
On the Applicability of Code Language Models to Scientific Computing Programs
Zhao, Q., Liu, F., Long, X., Wu, C., & Zhang, L. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3564599 · Cited by 1 · see citing works
SmartNote: An LLM-Powered, Personalised Release Note Generator That Just Works
Daneshyan, F., He, R., Wu, J., & Zhou, M. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3729345 · Cited by 2 · see citing works
Fan, F., Hao, J., & Fu, X. (2026) · Vehicles · doi:10.3390/vehicles8010005 · Cited by 1 · see citing works
Zhou, H., Peng, X., Kendre, S., Ryoo, M. S., Savarese, S., Xiong, C., & Niebles, J. C. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00451 · Cited by 1 · see citing works
Yu, N., Davies, L., & Zhong, W. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825828 · Cited by 1 · see citing works
Optimizing Telco Networks with Foundation Models: A Scalable Approach for Tabular Data
Le, F., Grueneberg, K., Nahum, E., et al. (2025) · IEEE International Conference on Distributed Computing Systems Workshops (ICDCSW) · doi:10.1109/icdcsw63273.2025.00043
DriveCode: Domain Specific Numerical Encoding for LLM-Based Autonomous Driving
Wang, Z., Jiang, Y., Zhou, R., et al. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3726335
Can Large Language Models Handle Numeric Constraints? A Comprehensive Study and Solutions
Wang, H., Binru, Z., Xu, Z., Zhou, H., Yang, L., Xu, W., Zhu, J., & Wang, H. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-7078-2_16
Data‐Efficient Cycle‐Level Capacity Prediction Using 1D Deep Convolutional Network
Huang, T., Sun, B., Chen, Z., Zhao, C., Bakker, E., Lei, Y., Zhao, L., Wang, S., Liu, C., He, V. X., et al. (2026) · Advanced Intelligent Discovery · doi:10.1002/aidi.70153
- Authors
- Schwartz, E., Choshen, L., Shtok, J., Doveh, S., Karlinsky, L., & Arbelle, A.
- Affiliations
- IBM (United States) (US)
- Venue
- Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
- Published
- 2024
- Pages
- 206-212
- OpenAlex
- W4404783351
- Cited by
- 14
- References
- 0
- Citations by year
- 2026: 4 · 2025: 7 · 2024: 3
- FWCI
- 4.04
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- number encoding, numerical reasoning, large language models, place value, chain-of-thought, arithmetic exercises, number generation, natural language understanding, MMLU
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
