The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.
Let's Verify Step by Step
Open accessAbstract
In recent years, large language models have greatly improved in their ability to perform complex multi-step reasoning. However, even state-of-the-art models still regularly produce logical mistakes. To train more reliable models, we can turn either to outcome supervision, which provides feedback for a final result, or process supervision, which provides feedback for each intermediate reasoning step. Given the importance of training reliable models, and given the high cost of human feedback, it is important to carefully compare the both methods. Recent work has already begun this comparison, but many questions still remain. We conduct our own investigation, finding that process supervision significantly outperforms outcome supervision for training models to solve problems from the challenging MATH dataset. Our process-supervised model solves 78% of problems from a representative subset of the MATH test set. Additionally, we show that active learning significantly improves the efficacy of process supervision. To support related research, we also release PRM800K, the complete dataset of 800,000 step-level human feedback labels used to train our best reward model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Constitutional AI (Bai et al., 2022) uses pre-specified rules; reward design methods (PBRS (Ng et al., 1999), process rewards (Lightman et al., 2023), rule-based rewards (Shao et al., 2024)) encode L 3 -type knowledge but are humandesigned, not learned.”
From Experience Compression Spectrum: Unifying Memory, Skills, and Rules in LLM Agents (2026) · cites this paper as Lightman et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“current multi-step agents remain limited in process-level control. Although they can generate subgoals or action sequences, they usually lack an explicit mechanism to determine whether the current step is consistent with the task context and whether it effectively supports the current subgoal.”
From A Closed-Loop Modular Language Agent with Step Verification and Local Correction for Multi-Step Task Solving (2026) · cites this paper as [9]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Cobbe et al. (2021); Lightman et al. (2023); Wang et al. (2023b) train a verifier or a critique model on a high-quality dataset to verify or refine LLM outputs, which can be used to provide feedback for correcting prediction errors.”
From Large Language Models Cannot Self-Correct Reasoning Yet (2023) · cites this paper as Lightman et al. (2023)
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Interestingly, while [186] did not find any major downstream performance improvement with their method, [97] used similar methodology but instead trained larger models on a larger feedback dataset coupled with evaluation on a more difficult task found segment-level feedback to be significantly more useful.”
From RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs (2026) · cites this paper as [97]
The cited source says what the claim attributes to it.
In the citing paper
“A more granular assessment can be particularly valuable because it (1) not only allows users to pinpoint where inaccuracies occur (Liu et al., 2021) but also (2) serves as a reward model for developers to refine their generative systems (Lightman et al., 2023).”
From FacTool: Factuality Detection in Generative AI - A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios (2023) · cites this paper as Lightman et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Lightman et al. [130], whose finding that process rewards outperform outcome rewards seeded today's process-reward-model (PRM) literature”
From Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops (2026) · cites this paper as [130]
The cited source says what the claim attributes to it.
In the citing paper
“Let's Verify Step by Step Lightman et al. [2023] showed that process-level signals outperform outcome-only supervision on mathematical reasoning”
From Process Supervision via Verbal Critique Improves Reasoning in Large Language Models (2026) · cites this paper as Lightman et al. [2023]
The cited source says what the claim attributes to it.
In the citing paper
“just one iteration of MAGICORE on Llama-3-8B already outperforms Best-of-k sampling (Lightman et al., 2023) by 3.2%”
From MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoning (2025) · cites this paper as Lightman et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Additionally, our dataset explores the more general case of prompting API-based LLMs, whereas PRM800K uses a generator LLM that was heavily fine-tuned for math-specific problems.”
From LLMs cannot find reasoning errors, but can correct them given the error location (2024) · cites this paper as Lightman et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“process reward models [26] extend this to step-level supervision over reasoning traces.”
From MemGym: a Long-Horizon Memory Environment for LLM Agents (2026) · cites this paper as [26]
The cited source says what the claim attributes to it.
In the citing paper
“Step-level supervision has also been shown to be beneficial for reasoning verification [9].”
From A Closed-Loop Modular Language Agent with Step Verification and Local Correction for Multi-Step Task Solving (2026) · cites this paper as [9]
The cited source says what the claim attributes to it.
In the citing paper
“In addition to Table 1, we also examine the accuracy at the 5 different difficulty levels explicitly defined by MATH-500 (Lightman et al., 2023) in Appendix B.2, where the observations remain consistent.”
From Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models (2025) · cites this paper as Lightman et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“When using model-generated chains of thought, the intermediate steps may not always be correct (Cobbe et al., 2021; Uesato et al., 2022; Lightman et al., 2023; Wang et al., 2023a).”
From The Llama 3 Herd of Models (2024) · cites this paper as Lightman et al., 2023
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Recent studies [35, 41] have explored this question with established math benchmarks by comparing the upper-bound capabilities (pass@k) of RL-based reasoning (thinking) models with their non-reasoning (non-thinking) standard LLM counterparts. They have shown that under equivalent inference token budgets, non-thinking LLMs can eventually reach performance comparable to thinking models on benchmarks”
From The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity (2025) · cites this paper as [42]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 23
- A. Askell, Y. Bai, A. Chen, D. Drain, D. Ganguli, T. Henighan, A. Jones, N. Joseph, B. Mann, N. DasSarma, et al. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861, 2021. · arXiv
- S. Bubeck, V. Chandrasekaran, R. Eldan, J. Gehrke, E. Horvitz, E. Kamar, P. Lee, Y. T. Lee, Y. Li, S. Lundberg, et al. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712, 2023. · arXiv
- P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017.
- K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. · arXiv
- A. Creswell, M. Shanahan, and I. Higgins. Selection-inference: Exploiting large language models for interpretable logical reasoning. arXiv preprint arXiv:2205.09712, 2022. · arXiv
- T. Everitt, V. Krakovna, L. Orseau, M. Hutter, and S. Legg. Reinforcement learning with a corrupted reward channel. arXiv preprint arXiv:1705.08417, 2017. · arXiv
- L. Gao, J. Schulman, and J. Hilton. Scaling laws for reward model overopti- mization. arXiv preprint arXiv:2210.10760, 2022. · arXiv
- D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. · arXiv
- T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa. Large language models are zero-shot reasoners. arXiv preprint arXiv:2205.11916, 2022. · arXiv
- A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. Ra- masesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, et al. Solving quantitative reasoning problems with language models. arXiv preprint arXiv:2206.14858, 2022. · arXiv
- Y. Li, Z. Lin, S. Zhang, Q. Fu, B. Chen, J.-G. Lou, and W. Chen. On the advance of making language models better reasoners. arXiv preprint arXiv:2206.02336, 2022. · arXiv
- J. Maynez, S. Narayan, B. Bohnet, and R. McDonald. On faithfulness and factuality in abstractive summarization. arXiv preprint arXiv:2005.00661, 2020. · arXiv
- R. Nakano, J. Hilton, S. Balaji, J. Wu, L. Ouyang, C. Kim, C. Hesse, S. Jain, V. Kosaraju, W. Saunders, et al. Webgpt: Browser-assisted question- answering with human feedback. arXiv preprint arXiv:2112.09332, 2021. · arXiv
- E. Nichols, L. Gao, and R. Gomez. Collaborative storytelling with large-scale neural language models. In Proceedings of the 13th ACM SIGGRAPH Con- ference on Motion, Interaction and Games, pages 1-10, 2020.
- M. Nye, A. J. Andreassen, G. Gur-Ari, H. Michalewski, J. Austin, D. Bieber, D. Dohan, A. Lewkowycz, M. Bosma, D. Luan, et al. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114, 2021. · arXiv
- OpenAI. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. · arXiv
- J. Shen, Y. Yin, L. Li, L. Shang, X. Jiang, M. Zhang, and Q. Liu. Generate & rank: A multi-task framework for math word problems. arXiv preprint arXiv:2109.03034, 2021. · arXiv
- N. Stiennon, L. Ouyang, J. Wu, D. Ziegler, R. Lowe, C. Voss, A. Radford, D. Amodei, and P. F. Christiano. Learning to summarize with human feed- back. Advances in Neural Information Processing Systems, 33:3008-3021, 2020.
- J. Uesato, N. Kushman, R. Kumar, F. Song, N. Siegel, L. Wang, A. Creswell, G. Irving, and I. Higgins. Solving math word problems with process-and outcome-based feedback. arXiv preprint arXiv:2211.14275, 2022. · arXiv
- X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, and D. Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022. · arXiv
- J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. Chi, Q. Le, and D. Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903, 2022. · arXiv
- E. Zelikman, Y. Wu, J. Mu, and N. Goodman. Star: Bootstrapping reason- ing with reasoning. Advances in Neural Information Processing Systems, 35: 15476-15488, 2022.
- D. M. Ziegler, N. Stiennon, J. Wu, T. B. Brown, A. Radford, D. Amodei, P. Christiano, and G. Irving. Fine-tuning language models from human pref- erences. arXiv preprint arXiv:1909.08593, 2019. Problem · arXiv
Cited by
Sorted by citations
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025
Toward expert-level medical question answering with large language models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S. R., Cole-Lewis, H., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03423-7 · Cited by 1,734
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970
Unleashing the potential of prompt engineering for large language models
CHEN, B. ‐., Zhang, Z., Langrené, N., & Zhu, S. (2025) · Patterns · doi:10.1016/j.patter.2025.101260 · Cited by 1,097
Hicks, M. T., Humphries, J., & Slater, J. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09775-5 · Cited by 456
Savage, T., Nayak, A., Gallo, R., Rangan, E., & Chen, J. H. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01010-1 · Cited by 265
Large Language Models Can Self-Improve
Huang, J., Gu, S. S., Hou, L., Wu, Y., Wang, X., Yu, H., & Han, J. (2022) · arXiv · doi:10.48550/arxiv.2210.11610 · Cited by 230
Pre-Trained Language Models for Text Generation: A Survey
Li, J., Tang, T., Zhao, W. X., Nie, J., & Wen, J.-R. (2024) · ACM Computing Surveys · doi:10.1145/3649449 · Cited by 383
Large Language Models Can Self-Improve
Huang, J., Gu, S., Hou, L., Wu, Y., Wang, X., Yu, H., & Han, J. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.67 · Cited by 230
LLMs cannot find reasoning errors, but can correct them given the error location
Tyen, G., Mansoor, H., Carbune, V., Chen, P., & Mak, T. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.826 · Cited by 156
Detecting and Preventing Hallucinations in Large Vision Language Models
Gunjal, A., Yin, J., & Bas, E. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29771 · Cited by 161
Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models
Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10896-y · Cited by 141
Datasets for large language models: a comprehensive survey
Liu, Y., Cao, J., Liu, C., Ding, K., & Jin, L. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11403-7 · Cited by 268
Large language models streamline automated machine learning for clinical studies
Tayebi Arasteh, S., Han, T., Lotfinia, M., Kuhl, C., Kather, J. N., Truhn, D., & Nebelung, S. (2024) · Nature Communications · doi:10.1038/s41467-024-45879-8 · Cited by 114
Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.09388 · Cited by 17,283
Large Language Models are Better Reasoners with Self-Verification
Weng, Y., Zhu, M., Xia, F., Li, B., He, S., Liu, S., Sun, B., Liu, K., & Zhao, J. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.167 · Cited by 110
Pardos, Z. A., & Bhandari, S. (2024) · PLOS ONE · doi:10.1371/journal.pone.0304013 · Cited by 97
“The 32% error rate is very close to the 27% recently reported by OpenAI using ChatGPT 4 to answer AP Calculus, Physics, and Chemistry questions [64].”
MethodOpen Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., Freedman, R., Korbak, T., Lindner, D., Freire, P., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.15217 · Cited by 101
“To address this, some works have trained LLMs to better solve multi-step math problems with process supervision [Uesato et al., 2022, Lightman et al., 2023].”
MethodPan, L., Saxon, M., Xu, W., Nathani, D., Wang, X., & Wang, W. Y. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00660 · Cited by 86
When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs
Kamoi, R., Zhang, Y., Zhang, N., Han, J., & Zhang, R. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00713 · Cited by 200
Harnessing LLMs for Temporal Data - A Study on Explainable Financial Time Series Forecasting
Yu, X., Chen, Z., & Lu, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-industry.69 · Cited by 89
RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMs
Chaudhari, S., Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K., Deshpande, A., & Castro da Silva, B. (2026) · ACM Computing Surveys · doi:10.1145/3743127 · Cited by 92
Muennighoff, N., Yang, Z., Shi, W., Li, X. L., Fei-Fei, L., Hajishirzi, H., Zettlemoyer, L., Liang, P., Candes, E., & Hashimoto, T. (2025) · Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2025.emnlp-main.1025 · Cited by 76
Towards Revealing the Mystery behind Chain of Thought: a Theoretical Perspective
Feng, G., Zhang, B., Gu, Y., Ye, H., He, D., & Wang, L. (2023) · arXiv · doi:10.48550/arxiv.2305.15408 · Cited by 69
ChatGPT hallucinating: can it get any more humanlike?
Siontis, K. C., Attia, Z. I., Asirvatham, S. J., & Friedman, P. A. (2023) · European Heart Journal · doi:10.1093/eurheartj/ehad766 · Cited by 62
Shojaee, P., Mirzadeh, I., Alizadeh, K., Horton, M., Bengio, S., & Farajtabar, M. (2025) · arXiv · doi:10.48550/arxiv.2506.06941 · Cited by 45
LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods
Li, H., Dong, Q., Chen, J., Su, H., Zhou, Y., Ai, Q., Ye, Z., & Liu, Y. (2024) · arXiv · doi:10.48550/arxiv.2412.05579 · Cited by 40
MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoning
Chen, J., Prasad, A., Saha, S., Stengel-Eskin, E., & Bansal, M. (2025) · Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2025.emnlp-main.1660 · Cited by 38
Large Language Models Cannot Self-Correct Reasoning Yet
Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Song, X., & Zhou, D. (2023) · arXiv · doi:10.48550/arxiv.2310.01798 · Cited by 712
“Cobbe et al. (2021); Lightman et al. (2023); Wang et al. (2023) train a verifier or a critique model on a high-quality dataset to verify or refine LLMs’ outputs.”
BackgroundSiepmann, R. M., Huppertz, M., Rastkhiz, A., Reen, M., Corban, E., Schmidt, C., Wilke, S., Schad, P., Yüksel, C., Kuhl, C. K., et al. (2024) · European Radiology · doi:10.1007/s00330-024-10727-2 · Cited by 39
On large language models safety, security, and privacy: A survey
Zhang, R., Li, H., Qian, X., Jiang, W., & Chen, H.-X. (2025) · Journal of Electronic Science and Technology · doi:10.1016/j.jnlest.2025.100301 · Cited by 50
Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
Burns, C., Izmailov, P., Kirchner, J. H., Baker, B., Gao, L., Aschenbrenner, L., Chen, Y., Ecoffet, A., Joglekar, M., Leike, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.09390 · Cited by 34
Self-citationDomain-specific chatbots for science using embeddings
Yager, K. G. (2023) · Digital Discovery · doi:10.1039/d3dd00112a · Cited by 35
Yao, Z., Parashar, A., Zhou, H., Jang, W. S., Ouyang, F., Yang, Z., & Yu, H. (2025) · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2025.naacl-long.538 · Cited by 27
Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
Snell, C., Lee, J., Xu, K., & Kumar, A. (2024) · arXiv · doi:10.48550/arxiv.2408.03314 · Cited by 27
Chern, I.-C., Chern, S., Chen, S., Yuan, W., Feng, K., Zhou, C., He, J., Neubig, G., & Liu, P. (2023) · arXiv · doi:10.48550/arxiv.2307.13528 · Cited by 29
“A more granular assessment can be particularly valuable because it (1) not only allows users to pinpoint where inaccuracies occur (Liu et al., 2021) but also (2) serves as a reward model for developers to refine their generative systems (Lightman et al., 2023).”
BackgroundAlam, M. F., Lentsch, A., Yu, N., Barmack, S., Kim, S., Acemoğlu, D., Hart, A. J., Johnson, S., & Ahmed, F. (2024) · doi:10.21428/e4baedd9.e39b392d · Cited by 31
Critique of impure reason: Unveiling the reasoning behaviour of medical large language models
Sim, S. Z. Y., & Chen, T. (2025) · eLife · doi:10.7554/elife.106187 · Cited by 29
Duan, J. (2024) · International Journal of New Developments in Education · doi:10.25236/ijnde.2024.060205 · Cited by 37
Repeatability of Fine-Tuning Large Language Models Illustrated Using QLoRA
Alahmari, S. S., Hall, L., Mouton, P. R., & Goldgof, D. B. (2024) · IEEE Access · doi:10.1109/access.2024.3470850 · Cited by 30
Transforming agency: On the mode of existence of large language models
Barandiaran, X. E., & Almendros, L. S. (2025) · Phenomenology and the Cognitive Sciences · doi:10.1007/s11097-025-10094-3 · Cited by 35
Nakamoto, R., Flanagan, B., Yamauchi, T., Dai, Y., Takami, K., & Ogata, H. (2023) · Computers · doi:10.3390/computers12110217 · Cited by 18
Agent Lumos: Unified and Modular Training for Open-Source Language Agents
Yin, D., Brahman, F., Ravichander, A., Chandu, K., Chang, K.-W., Choi, Y., & Lin, B. Y. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.670 · Cited by 17
Generation Next: Experimentation with AI
Charness, G., Jabarian, B., & List, J. A. (2023) · National Bureau of Economic Research · doi:10.3386/w31679 · Cited by 15
Muennighoff, N., Yang, Z., Shi, W., Li, X. L., Fei-Fei, L., Hajishirzi, H., Zettlemoyer, L., Liang, P., Candès, E., & Hashimoto, T. (2025) · arXiv · doi:10.48550/arxiv.2501.19393 · Cited by 12
“[...] This confirms the energy spectrum is: ) .”
Yager, K. G. (2024) · Digital Discovery · doi:10.1039/d4dd00178h · Cited by 12
Knowledge-Centered Dual-Process Reasoning for Math Word Problems With Large Language Models
Liu, J., Huang, Z., Liu, Q., Ma, Z., Zhai, C., & Chen, E. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3556367 · Cited by 13
Agarwal, R., Singh, A., Zhang, L. M., Bohnet, B., Rosias, L., Chan, S., Zhang, B., Anand, A., Abbas, Z., Nova, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2404.11018 · Cited by 35
“We use the MATH500 test set from Light-4 man et al. [32] to report performance, and our 4-shot MATH prompt for data generation can be found in Figure A.12.”
Large Language Models Can Self-Correct with Key Condition Verification
Wu, Z., Zeng, Q., Zhang, Z., Tan, Z., Shen, C., & Jiang, M. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.714 · Cited by 36
“Lightman et al. (2023) constructed a large dataset with step-wise correctness labels from human annotators, and fine-tuned a GPT-4 (OpenAI, 2024) model on it.”
MethodLehmann, J., Bhandiwad, D., Gattogi, P., & Vahdati, S. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00671 · Cited by 13
The Hallucination Tax of Reinforcement Finetuning
Song, L., Shi, T., & Zhao, J. (2025) · Findings of the Association for Computational Linguistics: EMNLP 2025 · doi:10.18653/v1/2025.findings-emnlp.112 · Cited by 10
Legally Enforceable Hate Speech Detection for Public Forums
Luo, C., Bhambhoria, R. V., Dahan, S., & Zhu, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.730 · Cited by 10
Democratizing Reasoning Ability: Tailored Learning from Large Language Model
Wang, Z., Huang, S., Liu, Y., Wang, J., Song, M., Zhang, Z., Huang, H., Wei, F., Deng, W., Sun, F., et al. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.120 · Cited by 22
Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Survey
Winata, G. I., Zhao, H., Das, A., Tang, W., Yao, D. D. W., Zhang, S.-X., & Sahu, S. (2025) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.17541 · Cited by 30
Three Questions Concerning the Use of Large Language Models to Facilitate Mathematics Learning
Yen, A.-Z., & Hsu, W.-L. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.201 · Cited by 15
Meta-Cognitive Memory Policy Optimization for Long-Horizon LLM Agents
Liu, Z., Hao, Z., Chen, Y., Wang, H., Hou, J., Ding, R., Yang, Y., Ji, W., Xia, W., & Liu, F. (2026) · arXiv · Cited by 12
Revolutionizing High School Physics Education: A Novel Dataset
Anand, A., Addala, K., Baghel, K. K. S., Goel, A., Hira, M., Gupta, R., & Shah, R. R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-49601-1_5 · Cited by 8
Evaluation of Medium-large Language Models at Zero-shot Closed Book Generative Question Answering
Peinl, R., & Wirth, J. (2024) · doi:10.5121/csit.2024.140106 · Cited by 8
GPT-in-the-Loop: Supporting Adaptation in Multiagent Systems
do Nascimento, N. M., Alencar, P., & Cowan, D. D. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386490 · Cited by 22
XAI for medicine by ChatGPT code interpreter
Kitamura, K., Irvan, M., & Shigetomi Yamaguchi, R. (2023) · Proceedings of the 2023 5th International Conference on Big-data Service and Intelligent Computation · doi:10.1145/3633624.3633629 · Cited by 5
On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlook
Fan, M., Wang, C., Chen, C., Liu, Y., & Huang, J. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02375-w · Cited by 40
Assessing Hallucination in Large Language Models Under Adversarial Attacks
Tripathi, S., Griffith, H., & Rathore, H. (2024) · International Conference on Mobile and Secure Services (MobiSecServ) · doi:10.1109/mobisecserv63327.2024.10759969 · Cited by 5
Agent-as-a-Judge: Evaluate Agents with Agents
Zhuge, M., Zhao, C., Ashley, D., Wang, W., Khizbullin, D., Xiong, Y., Liu, Z., Chang, E., Krishnamoorthi, R., Tian, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.10934 · Cited by 5
Gentopia.AI: A Collaborative Platform for Tool-Augmented LLMs
Xu, B., Liu, X., Shen, H., Han, Z., Li, Y., Yue, M., Peng, Z., Liu, Y., Yao, Z., & Xu, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2023.emnlp-demo.20 · Cited by 4
Conic10K: A Challenging Math Problem Understanding and Reasoning Dataset
Wu, H., Hui, W., Chen, Y., Wu, W., Tu, K., & Zhou, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.427 · Cited by 8
MedS³: Towards Medical Slow Thinking with Self-Evolved Soft Dual-sided Process Supervision
Jiang, S., Liao, Y., Chen, Z., Zhang, Y., Wang, Y., & Wang, Y. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i37.40395 · Cited by 4
Free-MAD: Consensus-Free Multi-Agent Debate
Cui, Y., Fu, H., Zhang, H., Wang, L., & Zuo, C. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.1600 · Cited by 28
Faithful Self-Refinement in Mathematical Reasoning via Progressive Back-Translation
Liao, H., Zhu, Z., Hu, S., He, H., & Jin, Y. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890054 · Cited by 3
System Description: DeepLLM, Casting Dialog Threads into Logic Programs
Tarau, P. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-2300-3_7 · Cited by 4
Explore-Instruct: Enhancing Domain-Specific Instruction Coverage through Active Exploration
Wan, F., Huang, X., Yang, T., Quan, X., Bi, W., & Shi, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.587 · Cited by 13
Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
Sui, Y., Chuang, Y.-N., Wang, G., Zhang, J., Zhang, T., Yuan, J., Liu, H., Wen, A., Zhong, S., Zou, N., et al. (2025) · arXiv · Cited by 517
“For instance, DeepSeek-R1 [33] undergoes multiple rounds of SFT and RL training, emphasizing structured thinking templates and rule-based reward mechanisms.”
Empowering Large Language Model Agent through Step-Level Self-Critique and Self-Training
Zhai, Y., Liu, H., Zhang, Z., Lin, T., Xu, K., Yang, C., Feng, D., Ding, B., & Wang, H. (2025) · Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3726302.3729965 · Cited by 3
Selective Propositional Reasoning: A Cognitive Load-Aware Strategy for Enhanced Reasoning
Yue, Y., Lei, Y., Shi, W., & Zhou, Y. (2023) · International Conference on Cloud Computing, Big Data Application and Software Engineering (CBASE) · doi:10.1109/cbase60015.2023.10439142 · Cited by 2
Evaluation of Medium-Sized Language Models in German and English Language
Peinl, R., & Wirth, J. (2024) · International Journal on Natural Language Computing · doi:10.5121/ijnlc.2024.13101 · Cited by 2
Cultural and technological change in the future information environment
Slapakova, L., Fraser, A., Hughes, M., Aquilino, M., & Thue, K. (2024) · RAND Corporation eBooks · doi:10.7249/rra2662-1 · Cited by 2
ArithmeticGPT: empowering small-size large language models with advanced arithmetic skills
Liu, Z., Zheng, Y., Yin, Z., Chen, J., Liu, T., Mi, T., & Luo, W. (2025) · Machine Learning · doi:10.1007/s10994-024-06681-1 · Cited by 2
MiMo: Unlocking the Reasoning Potential of Language Model - From Pretraining to Posttraining
Xiaomi, L.-C., :, Xia, B., Shen, B., Cici, Zhu, D., Zhang, D., Wang, G., Zhang, H., Liu, H., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.07608 · Cited by 1
Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation
Krishna, S., Krishna, K., Mohananey, A., Schwarcz, S., Stambler, A., Upadhyay, S., & Faruqui, M. (2024) · arXiv · doi:10.48550/arxiv.2409.12941 · Cited by 1
From Complex to Simple: Unraveling the Cognitive Tree for Reasoning with Small Language Models
Yan, J., Wang, C., Zhang, T., He, X., Huang, J., & Zhang, W. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.828 · Cited by 5
ELWARD : Empowering Language Model With World Insights and Human‐Aligned Reward Design
Yong-ping, D., Li, S., Yan, R., Hou, Y., & Han, H. (2025) · Expert Systems · doi:10.1111/exsy.70055 · Cited by 1
Embodied Reasoning with Self-Feedback
Kak, P., & Jain, S. (2024) · IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT) · doi:10.1109/conecct62155.2024.10677317 · Cited by 1
Artificial Intelligence in Math Education of Engineers
Schönle, M. (2025) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon62633.2025.11016373 · Cited by 1
Advance Toward Artificial Superintelligence with OpenAI's O1 Reinforcement Learning and Ethics
Mirindi, D., Sinkhonde, D., Mirindi, F., & Bezabih, T. (2025) · International Conference on Artificial Intelligence, Robotics and Control (AIRC) · doi:10.1109/airc64931.2025.11077494 · Cited by 1
Calibrating Conservatism for Scalable Oversight
Overman, W., & Bayati, M. (2026) · arXiv · doi:10.48550/arxiv.2605.28807
“…demonstrates that written principles can partially substitute for continuous human judgment by enabling AI-generated feedback, while process supervision (Lightman et al., 2024) shows that supervising intermediate reasoning steps can outperform outcome-only supervision in mathematical reasoning.”
ARS: Adaptive Reasoning Suppression for Efficient Large Reasoning Language Models
Zheng, D. (2025) · arXiv · doi:10.48550/arxiv.2510.00071
“The adaptive mechanism ensures that suppression occurs only when true certainty exceeds the optimal threshold, with the error term diminishing as checkpoints increase. policy ← CoDFastPolicy(drafts=2, per_draft=10) soning benchmarks including MATH500 [12] and GSM8K.”
Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models
Liu, R., Sun, Y., Zhang, M., Bai, H., Yu, X., Yu, T., Yuan, C., & Hou, L. (2025) · arXiv · doi:10.48550/arxiv.2504.04823
“In addition to Table 1, we also examine the accuracy at the 5 different difficulty levels explicitly defined by MATH-500 (Lightman et al., 2023) in Appendix B.2, where the observations remain consistent.”
Experience Compression Spectrum: Unifying Memory, Skills, and Rules in LLM Agents
Zhang, X., Wang, G., Cui, Y., Qiu, W., Li, Z., Zhu, B., & He, P. (2026) · arXiv
“Constitutional AI (Bai et al., 2022) uses pre-specified rules; reward design methods (PBRS (Ng et al., 1999), process rewards (Lightman et al., 2023), rule-based rewards (Shao et al., 2024)) encode L 3 -type knowledge but are human-designed, not learned.”
CROP: Token-Efficient Reasoning in Large Language Models via Regularized Prompt Optimization
Shah, D., Badhe, S., Kathrotia, N., & Tiwari, P. (2026) · arXiv
ReVISE: Learning to Refine at Test-Time via Intrinsic Self-Verification
Lee, H., Oh, S., Kim, J., Shin, J., & Tack, J. (2025) · arXiv · doi:10.48550/arxiv.2502.14565 · Cited by 26
“As shown in Figure 8, in MATH-500, ReVISE trained on Llama-3.”
InfluentialCritique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback
Zhang, X., Zhang, Y., Sun, H., Feng, K., Lu, C., Yang, C., & Meng, H. (2025) · arXiv · doi:10.48550/arxiv.2506.03106 · Cited by 92
“Many studies [11, 12, 13, 14, 15] primarily use critiques for evaluation, transforming them into numerical rewards for model improvement via RL algorithms such as Proximal Policy Optimization (PPO) [16] or Group Relative Policy Optimization (GRPO) [17].”
MemGym: a Long-Horizon Memory Environment for LLM Agents
Xu, W., Wang, Y., Mei, K., Liang, K., Wang, Z., Jin, M., Zhang, H., Zhang, S.-X., Hua, W., Sahu, S., et al. (2026) · arXiv · doi:10.48550/arxiv.2605.20833 · Cited by 11
“Reward models trained from human or automated feedback have become a central tool for post-training LLMs [30], and process reward models [26] extend this to step-level supervision over reasoning traces.”
Reason Popper-ly: Patching In-Context Reasoning with Inductive Logic Programming
Chen, Z., & Ma, M. (2026) · arXiv · doi:10.48550/arxiv.2607.23019
“More broadly, process-level supervision has been motivated by the observation that final-answer correctness alone provides only a weak signal for reasoning quality Lightman et al. [2023].”
Process Supervision via Verbal Critique Improves Reasoning in Large Language Models
Chen, H.-Y. (2026) · arXiv · doi:10.48550/arxiv.2604.21611 · Cited by 1
Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops
Chen, M., Wang, L., & Qu, B. (2026) · arXiv
Leung, E., Lumer, E., Feld, C., Huber, A., Subbiah, V. K., & Paul, K. (2026) · arXiv · doi:10.48550/arxiv.2607.08700
“Each LLM-judged criterion is an independent rubric judgment structurally similar to a process reward model (PRM) step [Lightman et al., 2024], a per-step judgment used as an intermediate reward signal, and the task is narrow enough that a well-specified rubric may compensate for model scale.”
Li, H., Feng, L., & Wang, D. (2026) · Electronics · doi:10.3390/electronics15102011
Li, Z., Li, Z., Shi, Y., Wang, R., Yang, J., Liu, Z., Wu, X., Li, A., Huang, C., Shang, E., et al. (2026) · arXiv · doi:10.48550/arxiv.2607.08964
Tsui, K. (2025) · arXiv · doi:10.48550/arxiv.2507.02778 · Cited by 29
Metadata
- Authors
- Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., & Cobbe, K.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2305.20050
- OpenAlex
- W4379089709
- Cited by
- 65
- References
- 0
- Citations by year
- 2026: 3 · 2025: 23 · 2024: 20 · 2023: 19
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- process supervision, multi-step reasoning, large language models, active learning (machine learning), reward modeling, mathematical problem solving, logical errors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
