Solving Quantitative Reasoning Problems with Language Models
Open accessAbstract
Language models have achieved remarkable performance on a wide range of tasks that require natural language understanding. Nevertheless, state-of-the-art models have generally struggled with tasks that require quantitative reasoning, such as solving mathematics, science, and engineering problems at the college level. To help close this gap, we introduce Minerva, a large language model pretrained on general natural language data and further trained on technical content. The model achieves state-of-the-art performance on technical benchmarks without the use of external tools. We also evaluate our model on over two hundred undergraduate-level problems in physics, biology, chemistry, economics, and other sciences that require quantitative reasoning, and find that the model can correctly answer nearly a third of them.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Wei et al. [4], Nye et al. [15] and Lewkowycz et al. [5] showed that combining prompting (in-context learning) with scratchpad strategies presents a powerful combination.
“Wei et al. [4], Nye et al. [15] and Lewkowycz et al. [5] showed that combining prompting (i.e. in-context learning) with scratchpad strategies present a powerful combination.”
From Exploring Length Generalization in Large Language Models (2022) · cites this paper as [5]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 10
- Alemi, A. A., Chollet, F., Een, N., Irving, G., Szegedy, C., and Urban, J. (2016). Deepmath - Deep Sequence Models for Premise Selection. In Proceedings of the 30th International Conference on Neural Information Processing Systems , NIPS'16, pages 2243-2251, USA. Curran Associates Inc.
- Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. Advances in neural information processing systems , 33:1877-1901.
- Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. d. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., Ryder, N., Pavlov, M., Power, A., Kaiser, L., Bavarian, M., Winter, C., Tillet, P., Such, F. P., Cummings, D., Plappert, M., Chantzis, F., Barnes, E., Herbert-Voss, A., Guss, W. H., Nichol, A., Paino, A., Tezak, N., Tang, J., Babuschkin, I., Balaji, S., Jain, S., Saunders, W., Hesse, C., Carr, A. N., Leike, J., Achiam, J., Misra, V., Morikawa, E., Radford, A., Knight, M., Brundage, M., Murati, M., Mayer, K., Welinder, P., McGrew, B., Amodei, D., McCandlish, S., Sutskever, I., and Zaremba, W. (2021). Evaluating large language models trained on code.
- Grabowski, A., Kornilowicz, A., and Naumowicz, A. (2010). Mizar in a nutshell. J. Formalized Reasoning , 3(2):153-245.
- Han, J. M., Rute, J., Wu, Y., Ayers, E., and Polu, S. (2022). Proof artifact co-training for theorem proving with language models. In International Conference on Learning Representations .
- Kaliszyk, C., Urban, J., Michalewski, H., and Olšák, M. (2018). Reinforcement learning of theorem proving. In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R., editors, Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc.
- Li, W., Yu, L., Wu, Y., and Paulson, L. C. (2021). Isarstep: a benchmark for high-level mathematical reasoning. In ICLR .
- Megill, N. D. and Wheeler, D. A. (2019). Metamath: A Computer Language for Pure Mathematics . Lulu Press, Morrisville, North Carolina. http://us.metamath.org/downloads/metamath.pdf .
- Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. (2019). Language models are unsupervised multitask learners. OpenAI blog , 1(8):9.
- Wu, Y., Rabe, M. N., Li, W., Ba, J., Grosse, R. B., and Szegedy, C. (2021). LIME: learning inductive bias for primitives of mathematical reasoning. In Meila, M. and Zhang, T., editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event , volume 139 of Proceedings of Machine Learning Research , pages 11251-11262. PMLR.
Cited by
Sorted by citations
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · see citing works
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · see citing works
“We also add a new subcorpora for arXiv and math webpages from the dataset used in Lewkowycz et al. (2022) in the proportion of 5%.”
MethodSelf-citationCode as Policies: Language Model Programs for Embodied Control
Liang, J., Huang, W., Xia, F., Xu, P., Hausman, K., Ichter, B., Florence, P., & Zeng, A. (2023) · 2023 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra48891.2023.10160591 · Cited by 767 · see citing works
Unleashing the potential of prompt engineering for large language models
Chen, B., Zhang, Z.-F., Langrené, N., & Zhu, S.-X. (2023) · Patterns · doi:10.1016/j.patter.2025.101260 · Cited by 462 · see citing works
Towards Reasoning in Large Language Models: A Survey
Huang, J., & Chang, K. C.-C. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.67 · Cited by 428 · see citing works
Becker, B. A., Denny, P., Finnie-Ansley, J., Luxton-Reilly, A., Prather, J., & Santos, E. A. (2023) · Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3545945.3569759 · Cited by 401 · see citing works
Large Language Models in Medicine: The Potentials and Pitfalls
Omiye, J. A., Gui, H., Rezaei, S. J., Zou, J., & Daneshjou, R. (2024) · Annals of Internal Medicine · doi:10.7326/m23-2772 · Cited by 394 · see citing works
Solving olympiad geometry without human demonstrations
Trinh, T. H., Wu, Y., Le, Q. V., He, H., & Luong, T. (2024) · Nature · doi:10.1038/s41586-023-06747-5 · Cited by 311 · see citing works
Self-citationA Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs)
Patil, R., & Gudivada, V. N. (2024) · Applied Sciences · doi:10.3390/app14052074 · Cited by 315 · see citing works
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
Brohan, A., Brown, N., Carbajal, J., Chebotar, Y., Chen, X., Choromanski, K., Ding, T., Driess, D., Dubey, A., Finn, C., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.15818 · Cited by 276 · see citing works
“…for a wide range of downstream tasks: large language models can enable not only fluent text generation [1, 2, 3] but emergent problem-solving [4, 5, 6] and creative generation of prose [7, 2] and code [8], while vision-language models enable open-vocabulary visual recognition [9, 10, 11] and…”
BackgroundSelf-citationLarge Language Models Can Self-Improve
Huang, J., Gu, S., Hou, L., Wu, Y., Wang, X., Yu, H., & Han, J. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.67 · Cited by 230 · see citing works
Self-citationAGIEval: A Human-Centric Benchmark for Evaluating Foundation Models
Zhong, W., Cui, R., Guo, Y., Liang, Y., Lu, S., Wang, Y., Saied, A., Chen, W., & Duan, N. (2023) · arXiv · doi:10.48550/arxiv.2304.06364 · Cited by 211 · see citing works
“For MATH, we use the same instances as in the appendices of Lewkowycz et al. (2022).”
Autonomous GIS: the next-generation AI-powered GIS
Li, Z., & Ning, H. (2023) · International Journal of Digital Earth · doi:10.1080/17538947.2023.2278895 · Cited by 201 · see citing works
Wang, M., Wang, M., Xu, X., Yang, L., Cai, D., & Yin, M. (2023) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2023.3324714 · Cited by 176 · see citing works
PAL: Program-aided Language Models
Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Y., Callan, J., & Neubig, G. (2022) · arXiv · doi:10.48550/arxiv.2211.10435 · Cited by 155 · see citing works
A Survey of Techniques for Optimizing Transformer Inference
Chitty-Venkata, K. T., Mittal, S., Emani, M., Vishwanath, V., & Somani, A. K. (2023) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2023.102990 · Cited by 154 · see citing works
Is GPT-3 a Good Data Annotator?
Ding, B., Qin, C., Liu, L., Chia, Y. K., Li, B., Joty, S., & Bing, L. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.626 · Cited by 149 · see citing works
Large Language Models Are Reasoning Teachers
Ho, N., Schmid, L., & Yun, S.-Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.830 · Cited by 238 · see citing works
Reasoning with Language Model Prompting: A Survey
Qiao, S., Ou, Y., Zhang, N., Chen, X., Yao, Y., Deng, S., Tan, C., Huang, F., & Chen, H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.294 · Cited by 218 · see citing works
What does CLIP know about a red circle? Visual prompt engineering for VLMs
Shtedritski, A., Rupprecht, C., & Vedaldi, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01101 · Cited by 142 · see citing works
Large Language Models as Evolutionary Optimizers
Liu, S., Chen, C., Qu, X., Tang, K., & Ong, Y.-S. (2024) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec60901.2024.10611913 · Cited by 154 · see citing works
Towards Faithful Model Explanation in NLP: A Survey
Lyu, Q., Apidianaki, M., & Callison-Burch, C. (2024) · Computational Linguistics · doi:10.1162/coli_a_00511 · Cited by 143 · see citing works
Faithful Chain-of-Thought Reasoning
Lyu, Q., Havaldar, S., Stein, A., Zhang, L., Rao, D., Wong, E., Apidianaki, M., & Callison-Burch, C. (2023) · Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.ijcnlp-main.20 · Cited by 201 · see citing works
Espejel, J. L., Ettifouri, E. H., Alassan, M. S. Y., Chouham, E. M., & Dahhane, W. (2023) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100032 · Cited by 131 · see citing works
Generative Artificial Intelligence for Chest Radiograph Interpretation in the Emergency Department
Huang, J., Neill, L., Wittbrodt, M., Melnick, D., Klug, M., Thompson, M., Bailitz, J., Loftus, T., Malik, S., Phull, A., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.36100 · Cited by 113 · see citing works
ParameterNet: Parameters are All You Need for Large-Scale Visual Pretraining of Mobile Networks
Han, K., Wang, Y., Guo, J., & Wu, E. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01491 · Cited by 120 · see citing works
Wang, F., Zhang, Z., Zhang, X., Wu, Z., Mo, T., Lu, Q., Wang, W., Li, R., Xu, J., Tang, X., et al. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3768165 · Cited by 106 · see citing works
Recent Advances in Large Language Models for Healthcare
Nassiri, K., & Akhloufi, M. A. (2024) · BioMedInformatics · doi:10.3390/biomedinformatics4020062 · Cited by 105 · see citing works
A Survey of Generative AI Applications
Gozalo-Brizuela, R., & Merchan, E. E. G. (2024) · Journal of Computer Science · doi:10.3844/jcssp.2024.801.818 · Cited by 101 · see citing works
A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets
Laskar, M. T. R., Bari, M. S., Rahman, M., Bhuiyan, M. A. H., Joty, S., & Huang, J. X. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.29 · Cited by 117 · see citing works
Improving Factuality and Reasoning in Language Models through Multiagent Debate
Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., & Mordatch, I. (2023) · arXiv · doi:10.48550/arxiv.2305.14325 · Cited by 96 · see citing works
“If the butter costs $4.00 a pound, how much will it cost him to purchase 6 pounds of butter? sider generating responses from multiple instances of model and performing majority voting across responses (Wang et al., 2022; Lewkowycz et al., 2022).”
InfluentialSurvey of Different Large Language Model Architectures: Trends, Benchmarks, and Challenges
Shao, M., Basit, A., Karri, R., & Shafique, M. (2024) · IEEE Access · doi:10.1109/access.2024.3482107 · Cited by 110 · see citing works
Baldur: Whole-Proof Generation and Repair with Large Language Models
First, E., Rabe, M. N., Ringer, T., & Brun, Y. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3616243 · Cited by 94 · see citing works
Towards provably efficient quantum algorithms for large-scale machine-learning models
Liu, J., Liu, M., Liu, J.-P., Ye, Z., Wang, Y., Alexeev, Y., Eisert, J., & Jiang, L. (2024) · Nature Communications · doi:10.1038/s41467-023-43957-x · Cited by 98 · see citing works
Turpin, M., Michael, J., Perez, E., & Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2305.04388 · Cited by 262 · see citing works
Impact of Pretraining Term Frequencies on Few-Shot Numerical Reasoning
Razeghi, Y., Logan IV, R. L., Gardner, M., & Singh, S. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.59 · Cited by 89 · see citing works
Li, X., Chan, S., Zhu, X., Pei, Y., Ma, Z., Liu, X., & Shah, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-industry.39 · Cited by 86 · see citing works
A Scoping Survey of ChatGPT in Mathematics Education
Pepin, B., Buchholtz, N. F., & Salinas-Hernández, U. (2025) · Digital Experiences in Mathematics Education · doi:10.1007/s40751-025-00172-1 · Cited by 138 · see citing works
Harnessing LLMs for Temporal Data - A Study on Explainable Financial Time Series Forecasting
Yu, X., Chen, Z., & Lu, Y.-B. (2023) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-industry.69 · Cited by 75 · see citing works
Distilling Reasoning Capabilities into Smaller Language Models
Shridhar, K., Stolfo, A., & Sachan, M. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.441 · Cited by 79 · see citing works
Few-shot In-context Learning on Knowledge Base Question Answering
Li, T., Ma, X., Zhuang, A., Gu, Y., Su, Y., & Chen, W. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.385 · Cited by 75 · see citing works
Do Large Language Models Know What They Don’t Know?
Yin, Z., Sun, Q., Guo, Q., Wu, J., Qiu, X., & Huang, X. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.551 · Cited by 75 · see citing works
Integration of large language models and federated learning
Chen, C., Feng, X., Li, Y., Lyu, L., Zhou, J., Zheng, X., & Yin, J. (2024) · Patterns · doi:10.1016/j.patter.2024.101098 · Cited by 103 · see citing works
Large Language Models Meet Next-Generation Networking Technologies: A Review
Hang, C. N., Yu, P.-D., Morabito, R., & Tan, C. W. (2024) · Future Internet · doi:10.3390/fi16100365 · Cited by 71 · see citing works
A Survey of Deep Learning for Mathematical Reasoning
Lu, P., Qiu, L., Yu, W., Welleck, S., & Chang, K.-W. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.817 · Cited by 84 · see citing works
AI Reasoning in Deep Learning Era: From Symbolic AI to Neural–Symbolic AI
Liang, B., Wang, Y., & Tong, C. (2025) · Mathematics · doi:10.3390/math13111707 · Cited by 79 · see citing works
Use of Artificial Intelligence Chatbots in Interpretation of Pathology Reports
Steimetz, E., Minkowitz, J. M., Gabutan, E. C., Ngichabe, J., Attia, H., Hershkop, M., Ozay, F., Hanna, M. G., & Gupta, R. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.12767 · Cited by 64 · see citing works
Gemini-the most powerful LLM: Myth or Truth
Islam, R., & Ahmed, I. (2024) · Information Communication Technologies Conference (ICTC) · doi:10.1109/ictc61510.2024.10602253 · Cited by 84 · see citing works
Is Education Losing the Race with Technology?
OECD (2023) · Educational research and innovation · doi:10.1787/73105f99-en · Cited by 83 · see citing works
Kolmogorov–Arnold PointNet: Deep learning for prediction of fluid fields on irregular geometries
Kashefi, A. (2025) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/j.cma.2025.117888 · Cited by 55 · see citing works
Unlearn What You Want to Forget: Efficient Unlearning for LLMs
Chen, J., & Yang, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.738 · Cited by 52 · see citing works
From Deep Learning to Rational Machines
Buckner, C. J. (2023) · doi:10.1093/oso/9780197653302.001.0001 · Cited by 55 · see citing works
A Survey on Symbolic Knowledge Distillation of Large Language Models
Acharya, K., Velasquez, A., & Song, H. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3428519 · Cited by 50 · see citing works
A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs)
Patil, R., & Gudivada, A. V. (2024) · doi:10.20944/preprints202402.0357.v1 · Cited by 296 · see citing works
Schorcht, S., Buchholtz, N. F., & Baumanns, L. (2024) · Frontiers in Education · doi:10.3389/feduc.2024.1386075 · Cited by 45 · see citing works
Federated and edge learning for large language models
Piccialli, F., Chiaro, D., Qi, P., Bellandi, V., & Damiani, E. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102840 · Cited by 49 · see citing works
Exploring Length Generalization in Large Language Models
Anil, C., Wu, Y., Andreassen, A., Lewkowycz, A., Misra, V., Ramasesh, V., Slone, A., Gur-Ari, G., Dyer, E., & Neyshabur, B. (2022) · arXiv · doi:10.48550/arxiv.2207.04901 · Cited by 44 · see citing works
Self-citationLearning Mathematics with Large Language Models
Matzakos, N., Doukakis, S., & Moundridou, M. (2023) · International Journal of Emerging Technologies in Learning (iJET) · doi:10.3991/ijet.v18i20.42979 · Cited by 51 · see citing works
Gandolfi, A. (2024) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-024-00403-3 · Cited by 52 · see citing works
Automated scholarly paper review: Concepts, technologies, and challenges
Lin, J., Song, J., Zhou, Z., Chen, Y., & Shi, X. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101830 · Cited by 37 · see citing works
A comprehensive overview of Generative AI (GAI): Technologies, applications, and challenges
Zhang, Z., Zhang, J., Zhang, X., & Mai, W. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129645 · Cited by 37 · see citing works
Babovic, Z. B., Bajat, B., Djokić, V., Đorđević, F., Drašković, D., Filipović, N., Furht, B., Gačić, N., Ikodinović, I. B., Ilić, M., et al. (2023) · Journal Of Big Data · doi:10.1186/s40537-023-00731-6 · Cited by 34 · see citing works
Limitations of Language Models in Arithmetic and Symbolic Induction
Jing, Q., Wang, H., Li, Z., Li, S., & Yan, X. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.516 · Cited by 29 · see citing works
TelecomRAG: Taming Telecom Standards with Retrieval Augmented Generation and LLMs
Yilma, G. M., Ayala‐Romero, J. A., García‐Saavedra, A., & Costa‐Pérez, X. (2024) · ACM SIGCOMM Computer Communication Review · doi:10.1145/3711992.3711996 · Cited by 29 · see citing works
Symbol tuning improves in-context learning in language models
Wei, J., Hou, L., Lampinen, A. K., Chen, X., Huang, D. Y., Tay, Y., Chen, X., Lu, Y., Zhou, D., Ma, T., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.61 · Cited by 45 · see citing works
AECR: Automatic attack technique intelligence extraction based on fine-tuned large language model
Chen, M., Zhu, K., Lu, B., Li, D., Yuan, Q., & Zhu, Y. (2024) · Computers & Security · doi:10.1016/j.cose.2024.104213 · Cited by 29 · see citing works
ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models
Chen, Z., Zhou, K., Zhang, B., Gong, Z., Zhao, X., & Wen, J.-R. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.985 · Cited by 30 · see citing works
Human I/O: Towards a Unified Approach to Detecting Situational Impairments
Liu, X., Li, J. N., Kim, D., Chen, X., & Du, R. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642065 · Cited by 26 · see citing works
Transcending Scaling Laws with 0.1% Extra Compute
Tay, Y., Wei, J., Chung, H. W., Tran, V., So, D. R., Shakeri, S., Garcia, X., Zheng, S., Rao, J., Chowdhery, A., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.91 · Cited by 41 · see citing works
Silva, D. D., Kaynak, O., El-Ayoubi, M., Mills, N., Alahakoon, D., & Manic, M. (2024) · IEEE Industrial Electronics Magazine · doi:10.1109/mie.2024.3382962 · Cited by 28 · see citing works
Exploring Effective Factors for Improving Visual In-Context Learning
Sun, Y.-P., Chen, Q., Wang, J., Wang, J.-D., & Li, Z.-C. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3554410 · Cited by 73 · see citing works
Chern, I.-C., Chern, S., Chen, S., Yuan, W., Feng, K., Zhou, C., He, J., Neubig, G., & Liu, P. (2023) · arXiv · doi:10.48550/arxiv.2307.13528 · Cited by 29 · see citing works
“…users to complete multiple tasks (e.g., question answering (Thop-pilan et al., 2022), code generation (Chen et al., 2021), math problem solving (Lewkowycz et al., 2022), and scientific literature generation (Taylor et al., 2022)) through a natural language interface (Liu et al., 2023) with both…”
BackgroundTestolin, A. (2024) · Applied Sciences · doi:10.3390/app14020744 · Cited by 24 · see citing works
EXIF as Language: Learning Cross-Modal Associations between Images and Camera Metadata
Zheng, C., Shrivastava, A., & Owens, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00671 · Cited by 25 · see citing works
Opportunities and Challenges of Foundation Models in Industrial Manufacturing
Ayyat, M., Osman, M., & Nadeem, T. (2025) · IEEE Access · doi:10.1109/access.2025.3589777 · Cited by 25 · see citing works
AI and the Future of Skills, Volume 2
OECD (2023) · Educational research and innovation · doi:10.1787/a9fe53cb-en · Cited by 38 · see citing works
Data Authenticity, Consent, and Provenance for AI Are All Broken: What Will It Take to Fix Them?
Longpre, S., Mahari, R., Obeng-Marnu, N., Brannon, W., South, T., Kabbara, J., & Pentland, A. (2024) · doi:10.21428/e4baedd9.a650f77d · Cited by 23 · see citing works
Tree-Based Representation and Generation of Natural and Mathematical Language
Scarlatos, A., & Lan, A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.205 · Cited by 21 · see citing works
REAL: Resilience and Adaptation using Large Language Models on Autonomous Aerial Robots
Tagliabue, A., Kondo, K., Zhao, T., Peterson, M. B., Tewari, C. T., & How, J. P. (2024) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc56724.2024.10885890 · Cited by 24 · see citing works
MultiTool-CoT: GPT-3 Can Use Multiple External Tools with Chain of Thought Prompting
Inaba, T., Kiyomaru, H., Cheng, F., & Kurohashi, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.130 · Cited by 19 · see citing works
Using large language models to accelerate communication for eye gaze typing users with ALS
Cai, S., Venugopalan, S., Seaver, K., Xiao, X., Tomanek, K., Jalasutram, S., Morris, M. R., Kane, S. K., Narayanan, A., MacDonald, R. L., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-53873-3 · Cited by 33 · see citing works
Let’s Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs
Aggarwal, P., Madaan, A., Yang, Y., & Mausam, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.761 · Cited by 29 · see citing works
Integration of cognitive tasks into artificial general intelligence test for large models
Qu, Y., Wei, C., Du, P., Che, W., Zhang, C., Ouyang, W., Bian, Y., Xu, F., Hu, B., Du, K., et al. (2024) · iScience · doi:10.1016/j.isci.2024.109550 · Cited by 18 · see citing works
A Survey of Reasoning with Foundation ModelsPreprint
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Liu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2023) · doi:10.31219/osf.io/ac4sp · Cited by 51 · see citing works
Self-citationCapturing advanced human cognitive abilities with deep neural networks
McClelland, J. L. (2022) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2022.09.018 · Cited by 18 · see citing works
Cordeiro, F. C., da Silva, P. F., Tessarollo, A., Freitas, C., de Souza, E. A., Gomes, D. D. S. M., Souza, R. R., & Coelho, F. C. (2024) · Computers & Geosciences · doi:10.1016/j.cageo.2024.105714 · Cited by 16 · see citing works
Self-Polish: Enhance Reasoning in Large Language Models via Problem Refinement
Xi, Z., Jin, S., Zhou, Y., Zheng, R., Gao, S., Liu, J., Gui, T., Zhang, Q., & Huang, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.762 · Cited by 23 · see citing works
Salikutluk, V., Koert, D., & Jäkel, F. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230081 · Cited by 15 · see citing works
Explaining Data Patterns in Natural Language with Language Models
Singh, C., Morris, J. X., Aneja, J., Rush, A. M., & Gao, J.-F. (2023) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/2023.blackboxnlp-1.3 · Cited by 21 · see citing works
Quantum many-body physics calculations with large language models
Pan, H., Mudur, N., Taranto, W., Tikhanovskaya, M., Venugopalan, S., Bahri, Y., Brenner, M. P., & Kim, E.-A. (2025) · Communications Physics · doi:10.1038/s42005-025-01956-y · Cited by 15 · see citing works
Fang, M., Wan, X., Lu, F., Xing, F., & Zou, K. (2025) · Scientific Data · doi:10.1038/s41597-025-05283-3 · Cited by 16 · see citing works
Stolfo, A., Belinkov, Y., & Sachan, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.435 · Cited by 14 · see citing works
CoLE: A collaborative legal expert prompting framework for large language models in law
Li, B., Fan, S., Zhu, S., & Wen, L.-J. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113052 · Cited by 14 · see citing works
Hou, R., Wang, X., Zhang, W.-Y., Song, Z., Wang, K., Chen, Y., Liu, J.-P., & Ruan, T. (2025) · IEEE journal of biomedical and health informatics · doi:10.1109/jbhi.2025.3529682 · Cited by 16 · see citing works
Towards Specification-Driven LLM-Based Generation of Embedded Automotive Software
Patil, M. S., Ung, G., & Nyberg, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-75434-0_9 · Cited by 12 · see citing works
Multimodal Fake News Detection Based on Chain-of-Thought Prompting Large Language Models
Xu, Y., Ge, J., Lyu, G., Li, G., & Li, H. (2024) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc54092.2024.10831252 · Cited by 12 · see citing works
Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding
Süzgün, M., Melas-Kyriazi, L., & Jurafsky, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.262 · Cited by 15 · see citing works
A Video Is Worth 4096 Tokens: Verbalize Story Videos To Understand Them In Zero Shot
Bhattacharyya, A., Singla, Y., Krishnamurthy, B., Shah, R. R., & Chen, C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.608 · Cited by 15 · see citing works
Generative AI and digital twin integrated intelligent process planning：A conceptual frameworkPreprint
Xu, Q., Zhou, G., Zhang, C., Chang, F., Cao, Y., & Zhao, D. (2023) · Research Square · doi:10.21203/rs.3.rs-3652246/v1 · Cited by 12 · see citing works
Li, Y., Yuan, P., Feng, S., Pan, B., Sun, B., Wang, X. X., Wang, H., & Li, K. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29821 · Cited by 14 · see citing works
- Authors
- Lewkowycz, A., Andreassen, A., Dohan, D., Dyer, E., Michalewski, H., Ramasesh, V., Slone, A., Anil, C., Schlag, I., Gutman-Solo, T., Wu, Y., Neyshabur, B., Gur-Ari, G., & Misra, V.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- Pages
- 3843-3857
- ISSN
- 1049-5258 (ISSN-L)
- arXiv
- 2206.14858
- OpenAlex
- W7133211296
- Cited by
- 358
- References
- 0
- Citations by year
- 2026: 55 · 2025: 53 · 2024: 67 · 2023: 173 · 2022: 10
- FWCI
- 287.45
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- neural information processing systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Mathematics › Applied Mathematics › Mathematics Education and Pedagogy
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- quantitative reasoning, language models, mathematical problem solving, large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
