Impact of Pretraining Term Frequencies on Few-Shot Reasoning
Open accessAbstract
Pretrained Language Models (LMs) have demonstrated ability to perform numerical reasoning by extrapolating from a few examples in few-shot settings. However, the extent to which this extrapolation relies on robust reasoning is unclear. In this paper, we investigate how well these models reason with terms that are less frequent in the pretraining data. In particular, we examine the correlations between the model performance on test instances and the frequency of terms from those instances in the pretraining data. We measure the strength of this correlation for a number of GPT-based language models (pretrained on the Pile dataset) on various numerical deduction tasks (e.g., arithmetic and unit conversion). Our results consistently demonstrate that models are more accurate on instances whose terms are more prevalent, in some cases above $70\%$ (absolute) more accurate on the top 10\% frequent terms in comparison to the bottom 10\%. Overall, although LMs exhibit strong performance at few-shot numerical reasoning tasks, our results raise the question of how much models actually generalize beyond pretraining data, and we encourage researchers to take the pretraining data into account when interpreting evaluation results.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Razeghi et al. show a correlation between frequency in training and test performance, supporting the pattern matching hypothesis.
“Razeghi et al. (2022) show a correlation between frequency in training and test performance, supporting the pattern matching hypothesis, while Nezhurina et al. (2024) and McCoy et al. (2023) demonstrate the fragility of reasoning in LLMs on simple tasks.”
From GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models (2024) · cites this paper as Razeghi et al. (2022)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Part of this paper’s reference list could not be split into single entries.
Research works printed but not linked: 12
- [557]557. [URL https://aclanthology.org/2020. emnlp-main.557 .](https://aclanthology.org/2020.emnlp-main.557)
- [2]2. Luccioni, A. and Viviano, J. What's in the box? an analysis of undesirable content in the Common Crawl corpus. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) , pp. 182-189, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.acl-short.24. URL https: //aclanthology.org/2021.acl-short.24 . · DOI · Paper page
- [3]3. McCoy, T., Pavlick, E., and Linzen, T. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pp. 3428-3448, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10.18653/v1/P19-1334. URL https://aclanthology.org/P19-1334 . · DOI · Paper page
- [4]4. Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., and Zettlemoyer, L. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837 , 2022. · arXiv
- [5]5. Nelson, B., Barreno, M., Chi, F. J., Joseph, A. D., Rubinstein, B. I. P., Saini, U., Sutton, C., Tygar, J. D., and Xia, K. Exploiting machine learning to subvert your spam filter. In Proceedings of the 1st Usenix Workshop on Large-Scale Exploits and Emergent Threats , LEET'08, USA, 2008. USENIX Association.
- [7]7. Petroni, F., Rockt¨ aschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., and Miller, A. Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) , pp. 2463-2473, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/D19-1250. URL https://aclanthology.org/D19-1250 . · DOI · Paper page
- [8]8. Poliak, A., Naradowsky, J., Haldar, A., Rudinger, R., and Van Durme, B. Hypothesis only baselines in natural language inference. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics , pp. 180-191, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653/v1/ S18-2023. URL https://aclanthology.org/ S18-2023 . · DOI
- [11]11. Romanov, A., De-Arteaga, M., Wallach, H., Chayes, J., Borgs, C., Chouldechova, A., Geyik, S., Kenthapadi, K., Rumshisky, A., and Kalai, A. What's in a name? Reducing bias in bios without access to protected attributes. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , pp. 4187-4195, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics. doi: 10.18653/v1/N19-1424. URL https://aclanthology.org/N19-1424 . · DOI
- [12]12. Rudinger, R., Naradowsky, J., Leonard, B., and Van Durme, B. Gender bias in coreference resolution. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) , pp. 814, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653/v1/N18-2002. URL https://aclanthology.org/N18-2002 . · DOI · Paper page
- [13]13. Saxton, D., Grefenstette, E., Hill, F., and Kohli, P. Analysing mathematical reasoning abilities of neural models. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019 . OpenReview.net, 2019. URL https://openreview. net/forum?id=H1gR5iR5FX .
- [14]14. Sharoff, S. Know thy corpus! robust methods for digital curation of web corpora. In Proceedings of the 12th Language Resources and Evaluation Conference , pp. 2453-2460, Marseille, France, May 2020. European Language Resources Association. ISBN 979-10-9554634-4. URL https://aclanthology.org/2020. lrec-1.298 .
- [15]15. Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., and Singh, S. AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pp. 42224235, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.emnlp-main. 346. URL https://aclanthology.org/2020. emnlp-main.346 . · DOI
Cited by
Sorted by citations
Self-Instruct: Aligning Language Models with Self-Generated Instructions
Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., & Hajishirzi, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.754 · Cited by 1,243 · see citing works
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022) · arXiv · doi:10.48550/arxiv.2202.12837 · Cited by 752 · see citing works
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.759 · Cited by 755 · see citing works
“Razeghi et al. (2022) show that in-context learning performance is highly correlated with term frequencies in the pretraining data.”
BackgroundTowards Reasoning in Large Language Models: A Survey
Huang, J., & Chang, K. C.-C. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.67 · Cited by 428 · see citing works
Mallen, A., Asai, A., Zhong, V., Das, R., Khashabi, D., & Hajishirzi, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.546 · Cited by 340 · see citing works
Demystifying Prompts in Language Models via Perplexity Estimation
Gonen, H., Iyer, S., Blevins, T., Smith, N., & Zettlemoyer, L. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.679 · Cited by 109 · see citing works
Data Contamination: From Memorization to Exploitation
Magar, I., & Schwartz, R. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2022.acl-short.18 · Cited by 88 · see citing works
Dentella, V., Günther, F., & Leivada, E. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2309583120 · Cited by 67 · see citing works
GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models
Mirzadeh, I., Alizadeh, K., Shahrokhi, H., Tuzel, O., Bengio, S., & Farajtabar, M. (2024) · arXiv · doi:10.48550/arxiv.2410.05229 · Cited by 57 · see citing works
“Razeghi et al. (2022)”
Exploring Length Generalization in Large Language Models
Anil, C., Wu, Y., Andreassen, A., Lewkowycz, A., Misra, V., Ramasesh, V., Slone, A., Gur-Ari, G., Dyer, E., & Neyshabur, B. (2022) · arXiv · doi:10.48550/arxiv.2207.04901 · Cited by 44 · see citing works
“Indeed, Razeghi et al. [2] showed that the performance of LLMs on mathematical calculations correlates with term frequency in the training data.”
BackgroundReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing
Liu, R., & Shah, N. B. (2023) · arXiv · doi:10.48550/arxiv.2306.00622 · Cited by 23 · see citing works
Testolin, A. (2024) · Applied Sciences · doi:10.3390/app14020744 · Cited by 24 · see citing works
What undergraduate students need to know and actually know about generative AI
Rismanchian, S., Babar, E. T. R., & Doroudi, S. (2026) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2026.100554 · Cited by 5 · see citing works
GPT for RCTs?: Using AI to determine adherence to reporting guidelines
Wrightson, J. G., Blazey, P., Moher, D., Khan, K. M., & Ardern, C. L. (2023) · doi:10.1101/2023.12.14.23299971 · Cited by 24 · see citing works
- Authors
- Razeghi, Y., Logan, R. L., Gardner, M., & Singh, S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2202.07206
- OpenAlex
- W4221151371
- Cited by
- 53
- References
- 0
- Citations by year
- 2026: 1 · 2025: 5 · 2024: 6 · 2023: 33 · 2022: 8
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2046873)
- Defense Advanced Research Projects Agency (N660011924033)
- Office of Naval Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- numerical reasoning, term frequency, pre-training data, generalization, arithmetic, unit conversion, extrapolation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
