CoChat
Sign Up
PreprintCited by 1Topic ModelingNatural Language Processing TechniquesExplainable Artificial Intelligence (XAI)

What is a Number, That a Large Language Model May Know It?

Open access
Marjieh, Raja, Veselovsky, Veniamin, Griffiths, Thomas L., Sucholutsky, Ilia·2025·arXiv
DOI 10.48550/arxiv.2502.01540arXiv 2502.01540
Open this paper in CoChat — free accountOpen PDF

Abstract

Numbers are a basic part of how humans represent and describe the world around them. As a consequence, learning effective representations of numbers is critical for the success of large language models as they become more integrated into everyday decisions. However, these models face a challenge: depending on context, the same sequence of digit tokens, e.g., 911, can be treated as a number or as a string. What kind of representations arise from this duality, and what are its downstream implications? Using a similarity-based prompting technique from cognitive science, we show that LLMs learn representational spaces that blend string-like and numerical representations. In particular, we show that elicited similarity judgments from these models over integer pairs can be captured by a combination of Levenshtein edit distance and numerical Log-Linear distance, suggesting an entangled representation. In a series of experiments we show how this entanglement is reflected in the latent embeddings, how it can be reduced but not entirely eliminated by context, and how it can propagate into a realistic decision scenario. These results shed light on a representational tension in transformer models that must learn what a number is from text input.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2502.01540

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Levenshtein distance has been applied to other contexts evaluating LLM outputs, such as code generation and numerical representations.

    “Levenshtein distance has also been applied to other contexts evaluating LLM outputs, such as code generation (Pan et al., 2025) or how LLM learn numerical representations (Marjieh et al., 2025).”

    From FLAWS: A Benchmark for Error Identification and Localization in Scientific Papers (2025) · cites this paper as Marjieh et al., 2025

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 50

  1. Alain, G. and Bengio, Y. Understanding intermediate layers using linear classifier probes, 2018. URL https:// arxiv.org/abs/1610.01644 . · arXiv
  2. Bai, X., Wang, A., Sucholutsky, I., and Griffiths, T. L. Measuring implicit bias in explicitly unbiased large language models. arXiv preprint arXiv:2402.04105 , 2024. · arXiv
  3. Belinkov, Y. Probing classifiers: Promises, shortcomings, and advances, 2021. URL https://arxiv.org/ abs/2102.12452 . · arXiv
  4. Binz, M. and Schulz, E. Using cognitive psychology to understand GPT-3. Proceedings of the National Academy of Sciences , 120(6):e2218523120, 2023.
  5. Cheyette, S. J. and Piantadosi, S. T. A unified account of numerosity perception. Nature human behaviour , 4(12): 1265-1272, 2020.
  6. Cunningham, H., Ewart, A., Riggs, L., Huben, R., and Sharkey, L. Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600 , 2023. · arXiv
  7. DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 , 2025. · arXiv
  8. Dehaene, S. The number sense: How the mind creates mathematics . Oxford University Press, 2011.
  9. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 , 2019. · arXiv
  10. Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783 , 2024. · arXiv
  11. Gao, L., la Tour, T. D., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., and Wu, J. Scaling and evaluating sparse autoencoders. arXiv preprint arXiv:2406.04093 , 2024. · arXiv
  12. Geiger, A., Wu, Z., Potts, C., Icard, T., and Goodman, N. D. Finding alignments between interpretable causal variables and distributed neural representations. arXiv preprint arXiv:2303.02536 , 2024. · arXiv
  13. Gurnee, W. and Tegmark, M. Language models represent space and time. arXiv preprint arXiv:2310.02207 , 2024. · arXiv
  14. Gurnee, W., Nanda, N., Pauly, M., Harvey, K., Troitskii, D., and Bertsimas, D. Finding neurons in a haystack: Case studies with sparse probing. arXiv preprint arXiv:2305.01610 , 2023. · arXiv
  15. Hanna, M., Liu, O., and Variengien, A. How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model. In Thirty-seventh Conference on Neural Information Processing Systems , 2023.
  16. Hurst, A., Lerer, A., Goucher, A. P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., Welihinda, A., Hayes, A., Radford, A., et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 , 2024. · arXiv
  17. Jaech, A., Kalai, A., Lerer, A., Richardson, A., El-Kishky, A., Low, A., Helyar, A., Madry, A., Beutel, A., Carney, A., et al. Openai o1 system card. arXiv preprint arXiv:2412.16720 , 2024. · arXiv
  18. Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., Casas, D. d. l., Hanna, E. B., Bressand, F., et al. Mixtral of experts. arXiv preprint arXiv:2401.04088 , 2024. · arXiv
  19. Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., Schiefer, N., Hatfield-Dodds, Z., DasSarma, N., Tran-Johnson, E., et al. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221 , 2022. · arXiv
  20. Levenshtein, V. Binary codes capable of correcting deletions, insertions, and reversals. Proceedings of the Soviet physics doklady , 1966.
  21. Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 , 2024. · arXiv
  22. Manning, C. D., Clark, K., Hewitt, J., Khandelwal, U., and Levy, O. Emergent linguistic structure in artificial neural networks trained by self-supervision. Proceedings of the National Academy of Sciences , 117(48):30046-30054, 2020.
  23. Marjieh, R., Jacoby, N., Peterson, J. C., and Griffiths, T. L. The universal law of generalization holds for naturalistic stimuli. Journal of Experimental Psychology: General , 153(3):573, 2024a.
  24. Marjieh, R., Sucholutsky, I., van Rijn, P., Jacoby, N., and Griffiths, T. L. Large language models predict human sensory judgments across six modalities. Scientific Reports , 14(1):21445, 2024b.
  25. McCoy, R. T., Yao, S., Friedman, D., Hardy, M. D., and Griffiths, T. L. Embers of autoregression show how large language models are shaped by the problem they are trained to solve. Proceedings of the National Academy of Sciences , 121(41):e2322420121, 2024a.
  26. McCoy, R. T., Yao, S., Friedman, D., Hardy, M. D., and Griffiths, T. L. When a language model is optimized for reasoning, does it still show embers of autoregression? An analysis of OpenAI o1. arXiv preprint arXiv:2410.01792 , 2024b. · arXiv
  27. McCulloch, W. S. What is a number, that a man may know it, and a man, that he may know a number. General Semantics Bulletin , 26(27):7-18, 1961.
  28. McCulloch, W. S. and Pitts, W. A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics , 5:115-133, 1943.
  29. Meng, K., Bau, D., Andonian, A., and Belinkov, Y. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems , 35:17359-17372, 2022.
  30. Miller, K. and Gelman, R. The child's representation of number: A multidimensional scaling analysis. Child development , pp. 1470-1479, 1983.
  31. Nanda, N., Chan, L., Lieberum, T., Smith, J., and Steinhardt, J. Progress measures for grokking via mechanistic interpretability. arXiv preprint arXiv:2301.05217 , 2023a. · arXiv
  32. Nanda, N., Lee, A., and Wattenberg, M. Emergent linear representations in world models of self-supervised sequence models. In Belinkov, Y., Hao, S., Jumelet, J., Kim, N., McCarthy, A., and Mohebbi, H. (eds.), Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP , pp. 16-30, 2023b. doi: 10.18653/v1/2023.blackboxnlp-1.2. · DOI · Paper page
  33. Nieder, A. and Dehaene, S. Representation of number in the brain. Annual review of neuroscience , 32(1):185-208, 2009.
  34. Nylund, K., Gururangan, S., and Smith, N. A. Time is encoded in the weights of finetuned language models. arXiv preprint arXiv:2312.13401 , 2023. · arXiv
  35. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research , 12:2825-2830, 2011.
  36. Piantadosi, S. T. A rational analysis of the approximate number system. Psychonomic bulletin & review , 23:877886, 2016.
  37. Piantadosi, S. T., Jara-Ettinger, J., and Gibson, E. Children's learning of number words in an indigenous farmingforaging group. Developmental science , 17(4):553-563, 2014.
  38. Rogers, A., Kovaleva, O., and Rumshisky, A. A primer in bertology: What we know about how bert works. Transactions of the Association for Computational Linguistics , 8:842-866, 2021.
  39. Shepard, R. N. The analysis of proximities: multidimensional scaling with an unknown distance function. i. Psychometrika , 27(2):125-140, 1962.
  40. Shepard, R. N. Multidimensional scaling, tree-fitting, and clustering. Science , 210(4468):390-398, 1980.
  41. Shepard, R. N. Toward a universal law of generalization for psychological science. Science , 237(4820):1317-1323, 1987.
  42. Stolfo, A., Belinkov, Y., and Sachan, M. A mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis. arXiv preprint arXiv:2305.15054 , 2023. · arXiv
  43. Tenenbaum, J. Rules and similarity in concept learning. Advances in neural information processing systems , 12, 1999.
  44. Tenenbaum, J. B. and Griffiths, T. L. Generalization, similarity, and Bayesian inference. Behavioral and brain sciences , 24(4):629-640, 2001.
  45. Tenney, I., Xia, P., Chen, B., Wang, A., Poliak, A., McCoy, R. T., Kim, N., Van Durme, B., Bowman, S. R., Das, D., et al. What do you learn from context? probing for sentence structure in contextualized word representations. arXiv preprint arXiv:1905.06316 , 2019. · arXiv
  46. Tian, K., Mitchell, E., Zhou, A., Sharma, A., Rafailov, R., Yao, H., Finn, C., and Manning, C. D. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. arXiv preprint arXiv:2305.14975 , 2023. · arXiv
  47. Tversky, A. and Hutchinson, J. Nearest neighbor analysis of psychological spaces. Psychological review , 93(1):3, 1986.
  48. Vicente, A. and Falkum, I. L. Polysemy. In Oxford research encyclopedia of linguistics . 2017.
  49. Webb, T., Holyoak, K. J., and Lu, H. Emergent analogical reasoning in large language models. Nature Human Behaviour , 7(9):1526-1541, 2023.
  50. Zhu, F., Dai, D., and Sui, Z. Language models encode the value of numbers linearly. In Rambow, O., Wanner, L., Apidianaki, M., Al-Khalifa, H., Eugenio, B. D., and Schockaert, S. (eds.), Proceedings of the 31st International Conference on Computational Linguistics , pp. 693-709, 2025.

Cited by

Sorted by citations

Authors
Marjieh, R., Veselovsky, V., Griffiths, T. L., & Sucholutsky, I.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2502.01540
arXiv
2502.01540
OpenAlex
W4407178893
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
number representation, large language models, Levenshtein distance, similarity judgments, latent space embedding, transformer models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review