Do NLP Models Know Numbers? Probing Numeracy in Embeddings
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Eric Wallace, Yizhong Wang, Sujian Li, Sameer Singh, Matt Gardner. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Figure 1 We train a probing model to decode a number from its word embedding over a random 80% of the integers from [-500, 500], e.g., '71' → 71.0. We plot the model's predictions for all numbers from [-2000, 2000]. The model accurately decodes numbers within the training range (in blue), i.e., pre-trained embeddings like GloVe and BERT capture numeracy. However, the probe fails to extrapolate to larger numbers (in red). The Char-CNN (e) and Char-LSTM (f) are trained jointly with the probing model.](https://figures.cochat.ai/paper-figures/153bcb5dac394f6194eb8e7aa3ffa6ae/do-nlp-models-know-numbers-probing-numeracy-in-embeddings-figure-1.png)

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 3×
“, 2018), BERT (Devlin et al., 2019), and GloVe (Pennington et al.”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 3×
“, 2018), BERT (Devlin et al., 2019), and GloVe (Pennington et al.”
BackgroundGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 4×
“We find that all widely-used pre-trained embed-dings, e.g., ELMo (Peters et al., 2018), BERT (De-vlin et al., 2019), and GloVe (Pennington et al., 2014), capture numeracy: number magnitude is present in the embeddings, even for numbers in the thousands.”
BackgroundMethodInfluentialDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · cited 3×
“We find that all widely-used pre-trained embed-dings, e.g., ELMo (Peters et al., 2018), BERT (De-vlin et al., 2019), and GloVe (Pennington et al., 2014), capture numeracy: number magnitude is present in the embeddings, even for numbers in the thousands.”
MethodSelf-citationSQuAD: 100,000+ Questions for Machine Comprehension of Text
Rajpurkar, P., Zhang, J., Lopyrev, K., & Liang, P. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1264 · Cited by 6,208 · cited 2×
“The dataset’s input-output format is a superset of SQuAD (Rajpurkar et al., 2016): the an... JaMarcus Russell completed a 91-yard touchdown pass to rookie wide receiver Chaz Schilens .”
MethodUnderstanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · cited 2×
“, influence functions (Koh and Liang, 2017; Brunet et al., 2019)).”
BackgroundBidirectional Attention Flow for Machine Comprehension
Seo, M. J., Kembhavi, A., Farhadi, A., & Hajishirzi, H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01603 · Cited by 1,445 · cited 2×
“As in typical span selection models (Seo et al., 2017), an LSTM reads the list of token embeddings, and a weight matrix and softmax function assign a probability to each index using the model’s hidden state.”
BackgroundMethodGraves, A., Wayne, G., & Danihelka, I. (2014) · arXiv · doi:10.48550/arxiv.1410.5401 · Cited by 1,058 · cited 2×
“Synthetic Numerical Tasks Similar to our synthetic numerical reasoning tasks, other work considers sorting (Graves et al., 2014), counting (Weiss et al., 2018), or decoding tasks (Trask et al., 2018).”
MethodAssessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies
Linzen, T., Dupoux, E., & Goldberg, Y. (2016) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00115 · Cited by 737 · cited 1×
“Khandel-wal et al. (2018) show how language models leverage context, while Linzen et al. (2016) demonstrate that language models understand subject-verb agreement.”
BackgroundLinguistic Knowledge and Transferability of Contextual Representations
Liu, N. F., Gardner, M., Belinkov, Y., Peters, M. E., & Smith, N. A. (2019) · Proceedings of the 2019 Conference of the North · doi:10.18653/v1/n19-1112 · Cited by 621 · cited 2×
“Probing Models Our probes of numeracy parallel work in understanding the linguistic capabilities (literacy) of neural models (Conneau et al., 2018; Liu et al., 2019).”
BackgroundSelf-citation
Research works printed but not linked: 5
- [1]1. Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2017. Finegrained anal- ysis of sentence embeddings using auxiliary predic- tion tasks. In ICLR.
- [13]13. Nelson F Liu, Matt Gardner, Yonatan Belinkov, Matthew Peters, and Noah A Smith. 2019. Linguis- tic knowledge and transferability of contextual rep- resentations. In NAACL.
- [14]14. Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2018. Ad- vances in pre-training distributed word representa- tions. In LREC.
- [24]24. Andrew Trask, Felix Hill, Scott Reed, Jack W. Rae, Chris Dyer, and Phil Blunsom. 2018. Neural arith- metic logic units. In NeurIPS.
- [29]29. Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoc V. Le. 2018. QANet: Combining local convolution with global self-attention for reading comprehen- sion. In ICLR.
Cited by
Items 201–286 of 286
Yuan, W., Lin, T., Jiang, Z., & Wang, S. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128911 · Cited by 3
Cash or Comfort? How LLMs Value Your Inconvenience
Cedro, M., Ichmoukhamedov, T., Goethals, S., He, Y., Hinns, J., & Martens, D. (2026) · Communications of the ACM · doi:10.1145/3799434 · Cited by 3
“In addition, their responses can shift in surprising ways with small changes in prompt wording, which could potentially create avenues for adversarial attacks.”
Hearing, not heeding: procedural acknowledgment and substantive influence in rulemaking
Love, A. (2026) · Journal of Public Administration Research and Theory · doi:10.1093/jopart/muag003 · Cited by 3
Dataset for Evaluation of Mathematical Reasoning Abilities in Russian
Nefedov, M. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-59082-6_10 · Cited by 4
On Masked Language Models for Contextual Link Prediction
Brayne, A., Wiatrak, M., & Corneil, D. (2022) · Workshop on Knowledge Extraction and Integration for Deep Learning Architectures: Deep Learning Inside Out (DeeLIO) · doi:10.18653/v1/2022.deelio-1.9 · Cited by 4
“Their work has been further extended to probing areas including semantic (Ettinger, 2020; Wallace et al., 2019), commonsense (Tamborrino et al., 2020; Forbes et al., 2019; Roberts et al., 2020), and linguistic (Lin et al., 2019; Tenney et al., 2019) knowledge.”
BackgroundGem: Gaussian Mixture Model Embeddings for Numerical Feature Distributions
Rauf, H. T., Bogatu, A., Paton, N. W., & Freitas, A. (2024) · Research Explorer (The University of Manchester) · doi:10.48550/arxiv.2410.07485 · Cited by 2
“Several other recent approaches adopt numerical reasoning methods to embed numerical data, each tailored to specific applications, including number decoding [28], automatic data generation [6], numerical attribute estimation [12, 15] and data-to-text generation [24].”
Sayed, F., Park, A., Sullivan, P. S., & Ge, Y. (2026) · JMIR Research Protocols · doi:10.2196/79966 · Cited by 2
“For fact validation, deterministic techniques, including semantic relevance and text similarity metrics (eg, ROUGE and BLEU), often failed due to their limited ability to capture negations [106] and numerical accuracy [107].”
Analyzing numeracy skills of upper primary graders in real-world problem contexts
Akmal, N., Idris, K., Fauziana, F., & Harun, J. (2025) · Alifmatika Jurnal Pendidikan dan Pembelajaran Matematika · doi:10.35316/alifmatika.2025.v7i1.108-123 · Cited by 2
Kondragunta, M. M., Chen, A., Slot, K., Weering, S., & Caselli, T. (2023) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2023.semeval-1.250 · Cited by 2
“Wallace et al. (2019) have shown that BERT-based models have difficulty working with numbers.”
BackgroundArithmetic-Based Pretraining Improving Numeracy of Pretrained Language Models
Petrak, D., Moosavi, N. S., & Gurevych, I. (2023) · Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/2023.starsem-1.42 · Cited by 10
“Recent works question whether these models meet this requirement out-of-the-box (Wallace et al., 2019; Zhang et al., 2020): Common pretraining objectives such as the denoising autoencoder of BART (Lewis et al., 2020), the masked language modeling objective of BERT (Devlin et al., 2019), or the…”
BackgroundResultA Numeral and Affective Knowledge Enhanced Network for Aspect-based Financial Sentiment Analysis
Qin, C., Yu, C., Yuan, M., & Chang, J. (2023) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai59109.2023.00139 · Cited by 2
“However, current NLP models mainly treat numerals as plain texts without grasping the basic numeric concepts like magnitude and category [6].”
BackgroundMethodInfluentialBetter than BERT but Worse than BaselinePreprint
Liu, B., Huang, J., Cai, X., & Church, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.05915 · Cited by 2
“Nevertheless, there are opportunities to improve BERT on numbers (Wallace et al., 2019), negation (Ettinger, 2020), and more.”
Investigating Numeracy Learning Ability of a Text-to-Text Transfer ModelPreprint
Pal, K. K., & Baral, C. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.04672 · Cited by 3
“, 2019) like GloVe, Word2Vec, FastText and contextual embeddings (Wallace et al., 2019) like BERT through probing tasks like numeration, magnitude comparison, addition, listmaximum.”
BackgroundMethodInfluentialA Survey of Parameters Associated with the Quality of Benchmarks in NLPPreprint
Mishra, S. R., Arunkumar, A., Bryan, C., & Baral, C. R. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.07566 · Cited by 3
“Language models often fail at numerical reasoning (Naik et al., 2018a; Wallace et al., 2019; Mishra et al., 2022b; Lu et al., 2022).”
BackgroundSentence embeddings and their relation with sentence structures
Simoulin, A. (2022) · theses.fr (ABES) · doi:10.70675/b5607013zca2az42fez89e2z605415c0c2d7 · Cited by 3
Lompo, B. A., & Le, T.-D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.18448 · Cited by 3
“While [19] claimed that most word embedding models could capture numeracy, several limitations persist.”
BackgroundMethodInterleaving Text and Number Embeddings to Solve Mathemathics ProblemsPreprint
Alberts, M., Gabrieli, G., & Morales, I. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.19353 · Cited by 3
“Despite their versatility in natural language tasks, accurate mathematical reasoning remains a challenge [8], especially when Chain-of-Thought prompting or a scratchpad are not employed [9].”
BackgroundNumDecoders at SemEval-2024 Task 7: FlanT5 and GPT enhanced with CoT for Numerical Reasoning
Gonzalez, A., Hossain, M. Z., & Junaed, J. A. (2024) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2024.semeval-1.183 · Cited by 1
“Numerical reasoning has been extensively studied across diverse contexts, including word embedding (Wallace et al., 2019; Naik et al., 2019; Sun-dararaman et al., 2020) and math word problems (Wang et al., 2018; Cobbe et al., 2021).”
BackgroundFine-grained entity disambiguation through numeric pattern awareness in transformer models
Jang, J., Kim, S.-M., Moon, H., Shin, S.-H., Yun, M., & Wiseman, C. (2025) · Complex & Intelligent Systems · doi:10.1007/s40747-025-01936-3 · Cited by 1
Chokphantavee, S., Chokphantavee, S., & Deepaisarn, S. (2025) · IEEE Access · doi:10.1109/access.2025.3639458 · Cited by 1
“…lies solely in the number of offenders, it can be implied that the underlying embedding model, the WangchanBERTa, cannot deal with the numerical information that well, in accordance with the research by [55], which discusses the shortfall of the sub-word model like BERT in encoding numbers.”
Large Language Models Are Stochastic Measuring Devices
Mallory, F. (2026) · Oxford University Press eBooks · doi:10.1093/9780191998317.003.0003 · Cited by 1
Joint Translation and Unit Conversion for End-to-end Localization
Dinu, G., Mathur, P., Federico, M., Lauly, S., & Al-Onaizan, Y. (2020) · International Conference on Spoken Language Translation (IWSLT) · doi:10.18653/v1/2020.iwslt-1.32 · Cited by 2
In China, What We Need is More Than Mathematics: A Literature Review of Numeracy
Hu, H. (2020) · Journal of Education Teaching and Social Studies · doi:10.22158/jetss.v2n3p24 · Cited by 1
“…2015, p. 34; Mahendru & Mahindru, 2014, p. 11) or the ability to understand and work with numbers (Lister, 2013; McGarry et al., 2016; Mensah, 2016; Osafehinti, 2008; Oxford University Press, 2012; Parvanta et al., 2017; Rothman et al., 2009; Spithourakis & Riedel, 2018; Wallace et al., 2019).”
BackgroundLearning number reasoning for numerical table-to-text generation
Feng, X., Gong, H., Chen, Y., Sun, Y., Qin, B., Bi, W., Liu, X., & Liu, T. (2021) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-021-01305-9 · Cited by 1
“Furthermore, our work is similar but different from math word problems studies [8, 14, 15, 25, 27, 28, 30].”
ResultDeep Reinforcement Learning for IoT Interoperability
Klöser, S., Kotstein, S., Reuben, R., Zerrer, T., & Decker, C. (2021) · ARENA2036 · doi:10.1007/978-3-662-62962-8_23 · Cited by 1
The Impact of Content Deletion on Tabular Data Similarity Using Contextual Word Embeddings
Pilaluisa, J., & Tomás, D. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-18050-7_24 · Cited by 1
Learning Non-linguistic Skills without Sacrificing Linguistic Proficiency
Sharma, M., Muralidhar, N., & Ramakrishnan, N. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.340 · Cited by 6
Discourse Information for Document-Level Temporal Dependency Parsing
Niu, J., Ng, V., Rees, E. E., de Montigny, S., & Penn, G. (2023) · Workshop on Computational Approaches to Discourse (CODI) · doi:10.18653/v1/2023.codi-1.10 · Cited by 3
“This outcome is also expected, as a BERT-based language model struggles with representing numbers (Wallace et al., 2019).”
BackgroundTraVLR: Now You See It, Now You Don’t! A Bimodal Dataset for Evaluating Visio-Linguistic Reasoning
Chow, K. J., Tan, S., & Kan, M. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.242 · Cited by 5
“Models are generally successful in the caption-only setting, corroborating Wallace et al. (2019)’s findings that numeracy is encoded in the embeddings of language-only models.”
BackgroundGenerative AI models for type 2 diabetes mellitus risk prediction
Johnson, E. S., Gandhi, U., Reddy, U. S., & Umapathy, M. (2025) · Health Systems · doi:10.1080/20476965.2025.2546904 · Cited by 1
“Prior works such as (Borisov et al., 2022; Dinh et al., 2022; Wallace et al., 2019) demonstrate that transformer-based language models can process numerical data by converting it into natural language sentences and leveraging their inherent text-under-standing capabilities.”
Enhancing the Numeracy of Word Embeddings: A Linear Algebraic Perspective
Ren, Y., & Du, Y. (2020) · Natural Language Processing and Chinese Computing · doi:10.1007/978-3-030-60450-9_14 · Cited by 3
Extend and Explain: Interpreting Very Long Language ModelsPreprint
Stremmel, J., Hill, B. L., Hertzberg, J., Murillo, J., Allotey, L., & Halperin, E. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.01174 · Cited by 4
“Numbers appearing in a healthcare context are relatively small, and it has been shown that Transformer LMs can e↵ectively represent the relative size of numbers in the range of -500 to 500 (Wallace et al., 2019).”
BackgroundExtraction and normalization of simple and structured entities in medical documents
Wajsbürt, P. (2021) · theses.fr (ABES) · doi:10.70675/2f777bcaz72aaz400ezb9f4za52428477079 · Cited by 18
LLM-based Semantic Search for Conversational Queries in E-commercePreprint
Siddiqui, E., Terikuti, V., & Lü, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.16492 · Cited by 2
“Second, embeddings often struggle to represent numerical values and categorical information [31], which may encode key requirements, as illustrated in the example query above.”
Eliciting Numerical Predictive Distributions of LLMs Without Autoregression
Piskorz, J., Kobalczyk, K., & van der Schaar, M. (2026) · International Conference on Learning Representations · doi:10.48550/arxiv.2603.02913 · Cited by 4
“Wallace et al. (2019) has shown that the value of a number can be successfully decoded from its encoded word embedding (e.g., “71” → 71 .”
Unveiling the Visual Counting Bottleneck in Vision-Language ModelsPreprint
Pang, X., Hou, Y., Wang, J., & Sachan, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.30170 · Cited by 2
“In contrast, neural models treat counting as a pattern-matching problem, degrading catastrophi-cally when object quantities exceed those observed during training (Wallace et al., 2019; Bender & Koller, 2020; Press et al., 2022; Anil et al., 2022).”
Cao, L., Chen, Q., & Guo, Y. (2026) · arXiv · doi:10.48550/arxiv.2601.21340
“…are primarily optimized for capturing semantic similarity and are known to be insensitive to precise numerical values and magnitude differences (Wallace et al., 2019; Zhang et al., 2020), which limits their effectiveness for reasoning over numeric clinical measurements such as laboratory…”
Natural Language Processing for Improving Transparency in Representative Democracy
Subramanian, S. (2020) · Minerva Access (University of Melbourne) · Cited by 1
Embedding Time Differences in Context-sensitive Neural Networks for Learning Time to Event
Dehghani, N., Hajipoor, H., & Amiri, H. (2021) · doi:10.18653/v1/2021.acl-short.80
“Motivated by recent research on neural numeracy learning (Chen et al., 2019; Wallace et al., 2019), we learn time difference embeddings–diff embeddings–as follows: we develop an LSTMbased character-level sequence to sequence model (based on the model presented in (Sutskever et al.”
MethodExploring the Use of Neural Transformers for Psycholinguistics
Laverghetta, A. (2021) · Digital Commons - University of South Florida (University of South Florida) · Cited by 1
Iconary: A Pictionary-Based Game for Testing Multimodal Communication with Drawings and TextPreprint
Clark, C., Salvador, J., Schwenk, D., Bonafilia, D., Yatskar, M., Kolve, E., Herrasti, A., Choi, J., Mehta, S., Skjonsberg, S., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.00800 · Cited by 1
“This gives the model some prior knowledge of what the icons are, and a sense of ordering among the quantized tokens (Wallace et al., 2019).”
BackgroundPre-training and evaluation of numeracy-oriented language model
Feng, F., Rui, X., Wang, W., Cao, Y., & Chua, T. (2021) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3490354.3494412
“Moreover, a few work has explored the evaluation of LM numeracy [26, 38, 44].”
BackgroundMethodNatural language processing on noisy text
Dong, R. (2021) · Digital Repository Service (Northeastern University) · doi:10.17760/d20416553
Predicting Numerals in Text Using Nearest Neighbor Language Models
Sakamoto, T., & Aizawa, A. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.295 · Cited by 1
“While humans can associate the magnitude of a numeral with the string of the numeral, LMs that treat numerals only as strings are unable to accurately make such associations for arbitrary numerals (Wallace et al., 2019).”
BackgroundMethodSulistyorini, T., Sova, E., Sofi, N., & Napitupulu, R. I. (2023) · International Journal Science and Technology · doi:10.56127/ijst.v2i3.995
“The maximum length value is determined from the longest sentence in the dataset and uses the padding type post, so padding will be added at the end of the sentence to equalize the length [18].”
BackgroundA Multi-modal Geospatial Encoding Network for Chinese Address Matching
Li, X., Wang, X., Wang, P., & Liu, M. (2024) · ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (SIGSPATIAL GIS) · doi:10.1145/3678717.3691243
Investigating the Performance of Dense Retrievers for Queries with Numerical Conditions
Fujimaki, H., & Kato, M. P. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88714-7_19
SCOPE: Compress Mathematical Reasoning Steps for Efficient Automated Process Annotation
Xu, H., Mao, X. P., Li, F.-L., Wu, X., Chen, W., Zhang, W. E., & Luu, A. T. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.1251 · Cited by 3
Pitch Class and Octave-Based Pitch Embedding Training Strategies for Symbolic Music Generation
Li, Y., Li, S., & Fazekas, G. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-02042-0_11
Self-citationBovo, R., Brudy, F., Fitzmaurice, G. W., & Anderson, F. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.08689
Reasoning Inconsistencies and How to Mitigate Them in Deep LearningPreprint
Arakelyan, E. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.02577
Developing trustworthy language models with consistent predictions
Jang, M. (2024) · Oxford University Research Archive (ORA) (University of Oxford) · doi:10.5287/ora-pm75v7jex
Exploring Lexical Sensitivities in Word Prediction Models: A case study on BERT
Misra, K. (2020) · Purdue e-Pubs (Purdue University System) · doi:10.25394/pgs.13308830
On Differential Privacy for Language Models
Arnold, S. (2024) · doi:10.25593/open-fau-1022
Recognizing refund values in complaints against brazilian teleommunication companies
Lira, L. D. L. (2021) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) · doi:10.34626/qgfr-n189
Domain-agnostic named entity recognition on unstructured text
Arora, J. (2021) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign)
Multi-objective Representation for Numbers in Clinical Narratives Using CamemBERT-bio
Aser Lompo, B., & LE, T.-D. (2024) · Open Repository and Bibliography (University of Luxembourg)
An investigation into neural arithmetic logic modules
Mistry, B. (2023) · ePrints Soton (University of Southampton) · Cited by 3
Leveraging Transformer Models and Elasticsearch to Help Prevent and Manage Diabetes through EFT Cues
Shah, A. (2023) · VTechWorks (Virginia Tech) · Cited by 1
Opening the Black Box: Empowering Machine Learning Models with Explanations
Setzu, M. (2022) · Electronic Theses and Dissertations Repository (University of Pisa)
Aser Lompo, B., & LE, T.-D. (2024) · Open Repository and Bibliography (University of Luxembourg)
Study of Pretraining Bias and Frequencies
Taware, R. (2023) · VTechWorks (Virginia Tech)
Silva, A. R. S. D. (2025) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
DNCC: Research on the Framework for Detecting Numerical Conflicts in Standard Clauses
Li, S., Zhang, H., Yang, G. J., Wang, X., Liao, F., Hou, Z., Wang, B., & Li, J. (2025) · IEEE International Conference on Artificial Intelligence, Computer Science and Information Processing (AICSIP) · doi:10.1109/aicsip65423.2025.11427315
Self-citationLearning with Limited Labeled Data
Ouali, Y. (2023) · theses.fr (ABES) · doi:10.70675/eede5c08zaa1bz4a68z941fzf8c8d07e9020 · Cited by 1
Towards automatic understanding of narrative audiovisual content
Reboud, A. (2022) · theses.fr (ABES) · doi:10.70675/7b1296fez9f91z4484z8448z8040d28ab833 · Cited by 1
Employing Transformers and Humans for Textual-Claim Verification
Saeed, M. (2022) · theses.fr (ABES) · doi:10.70675/3e02a3a6z8281z4d84z8f38ze617eaaf969b · Cited by 2
Granular Insights into Financial Discourse : Fine-Grained Opinion Analysis of Expert Texts
Hu, J. (2023) · theses.fr (ABES) · doi:10.70675/ed1d7748z8ebaz4550zbfb8z54bab2a6ac0a
Study of the multimodal understanding of vision-language transformer models
Salin, E. (2023) · theses.fr (ABES) · doi:10.70675/2beca704z5d3bz421dzbf61z61e00610aaeb
Classificação de Ideação Suicida em Postagens de Mídias Sociais
Vieira, H. L., & Amancio, D. R. (2025) · doi:10.11606/003296155
Beyond Hostility: Detecting Subtle and Overt Forms of Online Conflict with Multi-Objective Learning
Warke, O., Jose, J., & Breitsohl, J. (2026) · Information Retrieval Research · doi:10.54195/irrj.25366
“We adopted this scaling decision following research by Wallace et al. (2019), who investigate various NLP model’s understanding of numbers.”
Filho, A. D. S. L., Filho, A. K. D. B., Lima, F. S., dos Santos, S. M. L., & Sousa, R. B. V. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.19626
“Wallace et al. [24] establish, via probing, that standard embeddings capture magnitude only for integers up to three digits, collapsing for larger scales.”
Numerical Hallucinations in Retrieval-Augmented Generation: Detection and Analysis
Roy, S. S. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809882
“However, nearly one-third of numerical hallucinations remained undetected due to BERT’s subword tokenization, which fragments numbers into magnitude-unaware tokens [17].”
InfluentialLeveraging Large Language Models for Understanding Fundamental Principles of Catalysis
Michtavy, S. S., Perera, S. M. H. D., & Porosoff, M. D. (2026) · The Journal of Physical Chemistry C · doi:10.1021/acs.jpcc.6c02607 · Cited by 1
Probing the Intrinsic Effectiveness of Large Language Models for Medical Classifications
Lunardi, R., Roitero, K., & Mea, V. D. (2026) · ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB) · doi:10.1145/3807503.3819373
Generative Optimization for Incentivized Advertising with Global Level ConstraintsPreprint
Chen, G., Luo, N., Jiang, H., Li, D., Shu, W., Sha, T., Zeng, Y., Tai, W., Zhou, F., & Liu, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.04421
“Incentive allocation differs from language generation in that it involves continuous numerical values with strict magnitude constraints, where existing tokenization [36] or bucketing meth-ods [16, 38] fail to preserve numerical precision.”
Generative Optimization for Incentivized Advertising with Global Level Constraints
Chen, G., Luo, N., Jiang, H., Li, D., Shu, W., Sha, T., Zeng, Y. J., Tai, W., Zhou, F., & Liu, X. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3818423
“Incentive allocation differs from language generation in that it involves continuous numerical values with strict magnitude constraints, where existing tokenization [36] or bucketing meth-ods [16, 38] fail to preserve numerical precision.”
Priors, scaling, and redefinition of constants in LLMs
Antoniou, G. (2026) · DSpace - NTUA (National Technical University of Athens) · doi:10.26240/heal.ntua.33511
When Intelligence Must Stop: Artificial Intelligence, Representational Authority, and Lawful Closure
Huang, J. Y. (2026) · TSpace (University of Toronto)
A Numerically Aware GPT-Lite Method for Aluminum Alloy Composition Design
Hou, X., Liu, Y., Yin, J., & Zhou, H. (2026) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2026.089174
CCQ: A Multi-State Child Care Quality Dataset to Support AI for Children's Health ResearchPreprint
Li, V., Xie, Y., Dong, Z., Zhu, Q., Ma, W., Yang, C., Bai, J., Xu, H., & Lu, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.05863
“Prior work also shows that text models that split numbers into sub-word tokens represent numerical values less precisely [79].”
Yavuz, A., & SUBAŞI, İ. (2026) · İstanbul Medeniyet Üniversitesi Hukuk Fakültesi Dergisi · doi:10.58733/imhfd.1990358
Arnold, F., & Robert, J. B. (2024) · edoc Publication server (Humboldt University of Berlin) · doi:10.18452/39644
Detection and Exploration of References between Scholarly Texts and Literary Works
Arnold, F. (2026) · edoc Publication server (Humboldt University of Berlin) · doi:10.18452/39577
Workload-Aware, Traceable CPU Recommendation for HPC through Compiled NL-to-SQL
Aloulou, N., Jdidia, A. B., Baklouti, M., & Feki, S. (2026) · IEEE Symposium on Computers and Communications (ISCC) · doi:10.1109/iscc70364.2026.11740047
“Empirical evidence [3], [4] shows that standard embeddings struggle with numeracy and number-sensitive meaning, and dense single-vector retrieval has expressiveness limits that can miss certain relevance patterns even in simplified top-k settings [5].”
Metadata
- Authors
- Wallace, E., Wang, Y., Li, S., Singh, S., & Gardner, M.
- Affiliations
- Allen Institute for Artificial Intelligence (US)
- Peking University (CN)
- University of California, Irvine (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 5306-5314
- OpenAlex
- W2974596953
- Cited by
- 227
- References
- 28
- Citations by year
- 2026: 9 · 2025: 16 · 2024: 17 · 2023: 38 · 2022: 41 · 2021: 61 · 2020: 43 · 2019: 2
- FWCI
- 14.06
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Greater South Information System
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- numeracy, numerical reasoning, word embeddings, language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.