A Closed-Loop Modular Language Agent with Step Verification and Local Correction for Multi-Step Task Solving
Open accessThe DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index; 21 of the 33 sources its claims cite were checked for support.
Abstract
Multi-step task solving with large language models in intelligent electronic systems and interactive environments requires stronger process control and execution reliability. To address local error accumulation during multi-step execution, this paper proposes a closed-loop modular language agent framework integrating task planning, action execution, step verification, local regeneration, and replanning. A process-supervision data construction method is further introduced, in which real execution steps are retained as valid samples and invalid samples are automatically synthesized through action substitution, input perturbation, observation replacement, and subgoal mismatch, providing step-level supervision for validity prediction. In the proposed framework, step verification functions as a process-level control signal that supports hierarchical recovery through local regeneration and replanning, rather than as a standalone filtering module. Experiments are conducted on mathematical reasoning and web interaction tasks. On mathematical reasoning tasks, the proposed framework achieves an accuracy of 0.650, compared with 0.317 for Integrated Training, 0.460 for CoT Training, 0.568 for ReFT, and 0.617 for Agent Lumos. On web interaction tasks, the proposed framework achieves a step success rate of 0.424, compared with 0.246 for Integrated Training and 0.310 for Agent Lumos. Among the cases where recovery is triggered, local regrounding succeeds in 85.7% of reground attempts, while replanning succeeds in 53.3% of replanning attempts. These results indicate that the proposed framework improves process stability and recovery capability in multi-step task solving.
Overview
CoChat Summary
## TL;DR
The paper proposes CL-SPF, a closed-loop modular language agent that adds step verification and hierarchical recovery to multi-step task solving so invalid intermediate steps are corrected before they propagate. Experiments on mathematical reasoning and web interaction show improved performance over baseline methods, especially when verification is coupled with local regrounding and replanning.
## Key findings
- On mathematical reasoning tasks, CL-SPF achieved 0.650 accuracy, compared with 0.317 for Integrated Training, 0.460 for CoT Training, 0.568 for ReFT, and 0.617 for Agent Lumos.
- On web interaction tasks, CL-SPF achieved a 0.424 step success rate, compared with 0.246 for Integrated Training and 0.310 for Agent Lumos; the paper also reports 0.315 and 0.324 for Agent Lumos with fixed retries of R = 6 and R = 7.
- In the module ablation study on GSM, the full model achieved the highest Answer Accuracy (0.650); removing the verifier dropped accuracy to 0.567, removing regrounding to 0.608, and removing replanning to 0.633.
- For recovery, the paper reports 85.7% reground success and 53.3% replan success among triggered recovery attempts; on GSM, 55.0% of cases were solved after recovery was triggered, 32.5% remained unresolved after correction, and 10.0% of all samples were correct without triggering correction.
## Methods
- The framework is a closed-loop modular agent with four main modules: task planning, action execution, step verification, and recovery (local regeneration/regrounding, then replanning if needed).
- Experiments were run on mathematical reasoning datasets (GSM, ASDiv, PRM800K) and the web interaction dataset Mind2Web using its Test-Domain split.
- Step verification is trained as a binary classification problem using positive real execution steps and synthetic negative samples created by action substitution, input perturbation, observation replacement, and subgoal mismatch; the paper uses metrics at the task, step, and process levels.
## Limitations
- The paper states that the study is limited to two representative task settings and a fixed base model; future work will extend to broader domains and stronger models.
- It notes that verification and recovery add computational overhead through extra action attempts and replanning, and that direct wallclock latency and token consumption were not measured.
- It also states that the verifier is trained on automatically constructed negative samples, which may not fully cover real-world failure diversity.
- Evident from the text: several reported gains come with more action attempts / retries, so improved robustness is partly traded for extra execution cost.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
13 Valid · 0 Not valid · 9 Not checked · 136 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Once an intermediate action deviates from the current subgoal, the resulting error is easily propagated to subsequent steps and may ultimately reduce overall task performance [10,13,17]”
What the sources say
- Partly supportedcites [10] 10. Li, M.; Zhao, Y.; Yu, B.; Song, F.; Li, H.; Yu, H.; Li, Z.; Huang, F.; Li, Y. API-Bank: A comprehensive benchmark for tool-augmented LLMs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), Singapore, 6-10 December 2023; pp. 3102-3116. sourceExperimental — accuracy not yet measured
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
- Partly supportedcites [13] 13. Qin, Y.; Liang, S.; Ye, Y.; Zhu, K.; Yan, L.; Lu, Y.; Lin, Y.; Cong, X.; Tang, X.; Qian, B.; et al. ToolLLM: Facilitating Large Language Models to Master 16000+ Real-World APIs. In Proceedings of the Twelfth International Conference on Learning Representations (ICLR 2024), Vienna, Austria, 7-11 May 2024. sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
- Not matchedcites [17] 17. Wölflein, G.; Ferber, D.; Truhn, D.; Arandjelovi´ c, O.; Kather, J.N. LLM Agents Making Agent Tools. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vienna, Austria, 27 July-1 August 2025; pp. 26092-26130.
Its entry has no title and year that we can search by.
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“most existing feedback and verification methods remain focused on result-level correction [8,22], turn-level reflection [7], or auxiliary verification stages [19-21], with limited integration of step-level signals into the current action generation process”
What the sources say
- Supportedcites [8] 8. Gou, Z.; Shao, Z.; Gong, Y.; Shen, Y.; Yang, Y.; Duan, N.; Chen, W. CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing. In Proceedings of the Twelfth International Conference on Learning Representations (ICLR 2024), Vienna, Austria, 7-11 May 2024. source
The “Intrinsic Self-Correct with NL feedback” section says most methods in that line of research correct initial results. Sections 3.3–3.4 describe CRITIC generating an answer, verifying that output, and then using the critique to generate a corrected answer, supporting its classification as result-level correction rather than feedback integrated into the current generation step. (LLM, conf=0.87)
- Supportedcites [22] 22. Kumar, A.; Zhuang, V.; Agarwal, R.; Su, Y.; Co-Reyes, J.D.; Singh, A.; Baumli, K.; Iqbal, S.; Bishop, C.; Roelofs, R.; et al. Training Language Models to Self-Correct via Reinforcement Learning. In Proceedings of the Thirteenth International Conference on Learning Representations (ICLR 2025), Singapore, 24-28 April 2025. source
The citation is attached to “turn-level reflection.” In the Evaluation prompts section, the model is asked on its second attempt to assess and potentially rewrite its first response; the Experimental Evaluation section likewise describes two sequential attempts. That supports the cited paper as an example of turn-level correction, rather than correction integrated into the initial response as it is generated. (LLM, conf=0.88)
- Supportedcites [7] 7. Shinn, N.; Cassano, F.; Berman, E.; Gopinath, A.; Narasimhan, K.; Yao, S. Reflexion: Language Agents with Verbal Reinforcement Learning. Adv. Neural Inf. Process. Syst. 2023 , 36 , 8634-8652. source
Section 3 (pp. 3–4) describes Reflexion as evaluating a completed trajectory, generating self-reflection, and storing it for subsequent trials. The reflection can identify a specific mistaken action, but it informs later action choices rather than being integrated into generation of the current trajectory. This supports the portion of the sentence attributed to Reflexion [7]. (LLM, conf=0.88)
- Partly supportedcites [19] 19. Zhang, Y.; Khalifa, M.; Logeswaran, L.; Kim, J.; Lee, M.; Lee, H.; Wang, L. Small Language Models Need Strong Verifiers to Self-Correct Reasoning. In Proceedings of the Findings of the Association for Computational Linguistics: ACL 2024, Bangkok, Thailand, 11-16 August 2024; pp. 15637-15653. sourceExperimental — accuracy not yet measured
Our copy of SCORE does characterise the verification landscape in a way that covers part of the claim: the related-work paragraph 'Verifying Reasoning' distinguishes process/step-based verifiers (Uesato et al., 2022; Li et al., 2023b; Lightman et al., 2023) from outcome-based supervision (Cobbe et al., 2021; Yu et al., 2023a; Hosseini et al., 2024) and states the authors use the latter, and the Introduction (p. 1) notes prior self-correction work conflates SELF-VERIFY and SELF-REFINE, i.e. verification as a separate/auxiliary stage. However, the paper never says 'most existing methods' integrate step-level signals poorly, and it contains nothing about 'turn-level reflection' or about step-level signals feeding an 'action generation process' — its setting is chain-of-thought reasoning (GSM8K, CSQA, MATH), not agent action selection. Indeed Section 3.1 states the opposite tendency, that 'Step-level critiques are more useful than solution-level ones', and it explicitly credits an existing step/process-based verifier lineage. (LLM, conf=0.65)
- Partly supportedcites [20] 20. Wan, G.; Wu, Y.; Wang, H.; Zhao, S.; Chen, J.; Li, S. Derailer-Rerailer: Adaptive Verification for Efficient and Reliable Language Model Reasoning. In Proceedings of the Findings of the Association for Computational Linguistics: ACL 2025, Vienna, Austria, 27 July 2025; pp. 327-348. sourceExperimental — accuracy not yet measured
Our copy does characterize existing verification/feedback methods in a way that covers part of the claim: §2.1 (p. 3) splits iterative prompting into parallel sampling with aggregation/selection 'to obtain the final answer' (Self-Consistency, Best-of-N) and methods that 'sequentially evaluate and refine the reasoning path based on some evaluators' (Chain-of-Verification, Self-Refine) — i.e. result-level and auxiliary-verification orientations. However, the paper's own stated limitation of these methods is the accuracy–efficiency trade-off and their 'uniform application—indiscriminately deploying these expensive procedures across all queries' (1 Introduction, p. 1), not a granularity gap. The specific assertion that such methods have 'limited integration of step-level signals into the current action generation process' is not made in our copy; the paper only implicitly contrasts this by designing Rerailer to generate and pairwise-compare candidates per reasoning step during decoding (§3.2, pp. 5–6), and it explicitly credits CoVe with 'step-wise validation' (§4, p. 6), so the taxonomy framing and the step-level-integration gap are the unsupported parts. (LLM, conf=0.65)
- Could not verifycites [21] 21. Han, J.; Buntine, W.; Shareghi, E. VerifiAgent: A Unified Verification Agent in Language Model Reasoning. In Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2025, Suzhou, China, 4-9 November 2025; pp. 16410-16431. sourceExperimental — accuracy not yet measured
Our copy's related-work and introduction sections (2.1 LLMs as Verifiers; 1 Introduction) do survey prior feedback/verification work, but they characterise its limits on a different axis: 'domain restriction, computational inefficiency, and lack of scalability', shallow verification (Hong et al., 2024), unreliable self-critique without external grounding (Stechly et al., 2024), and the feature grid of Table 1 (training-free / generalised / tool-based / fine-grained feedback), plus higher cost of Process Reward Models. No passage frames existing methods as 'result-level correction, turn-level reflection, or auxiliary verification stages', and none discusses integrating step-level signals into the current action generation process — VerifiAgent itself verifies an already-produced solution in a two-layer meta + tool-based stage (3.1, 3.2, 3.3), and its step decomposition is inside verification, not action generation. Searched: step-level signals / action generation, post-hoc vs. during-generation verification, process reward models, limitations of existing verification methods; the claim's taxonomy is absent rather than contradicted. (LLM, conf=0.68)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On web interaction tasks, the proposed framework achieves a step success rate of 0.424, compared with 0.246 for Integrated Training and 0.310 for Agent Lumos.”
- Also stated in: § Abstract Multi-step task solving with large language models in intelligent electronic systems and interactive environments requires stronger process control and execution reliability. To address local error accumulation during multi-step execution, this paper proposes a closedloop modular language agent framework integrating task planning, action execution, step verification, local regeneration, and replanning. A process-supervision data construction method is further introduced, in which real execution steps are retained as valid samples and invalid samples are automatically synthesized through action substitution, input perturbation, observation replacement, and subgoal mismatch, providing step-level supervision for validity prediction. In the proposed framework, step verification functions as a process-level control signal that supports hierarchical recovery through local regeneration and replanning, rather than as a standalone filtering module. Experiments are conducted on mathematical reasoning and web interaction tasks. On mathematical reasoning tasks, the proposed framework achieves an accuracy of 0.650, compared with 0.317 for Integrated Training, 0.460 for CoT Training, 0.568 for ReFT, and 0.617 for Agent Lumos. On web interaction tasks, the proposed framework achieves a step success rate of 0.424, compared with 0.246 for Integrated Training and 0.310 for Agent Lumos. Among the cases where recovery is triggered, local regrounding succeeds in 85.7% of reground attempts, while replanning succeeds in 53.3% of replanning attempts. These results indicate that the proposed framework improves process stability and recovery capability in multi-step task solving. Keywords: closed-loop language agent; multi-step task solving; step verification; local correction; process supervision; large language models · plain
“On web interaction tasks, the proposed framework achieves a step success rate of 0.424”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Abstract Multi-step task solving with large language models in intelligent electronic systems and interactive environments requires stronger process control and execution reliability. To address local error accumulation during multi-step execution, this paper proposes a closedloop modular language agent framework integrating task planning, action execution, step verification, local regeneration, and replanning. A process-supervision data construction method is further introduced, in which real execution steps are retained as valid samples and invalid samples are automatically synthesized through action substitution, input perturbation, observation replacement, and subgoal mismatch, providing step-level supervision for validity prediction. In the proposed framework, step verification functions as a process-level control signal that supports hierarchical recovery through local regeneration and replanning, rather than as a standalone filtering module. Experiments are conducted on mathematical reasoning and web interaction tasks. On mathematical reasoning tasks, the proposed framework achieves an accuracy of 0.650, compared with 0.317 for Integrated Training, 0.460 for CoT Training, 0.568 for ReFT, and 0.617 for Agent Lumos. On web interaction tasks, the proposed framework achieves a step success rate of 0.424, compared with 0.246 for Integrated Training and 0.310 for Agent Lumos. Among the cases where recovery is triggered, local regrounding succeeds in 85.7% of reground attempts, while replanning succeeds in 53.3% of replanning attempts. These results indicate that the proposed framework improves process stability and recovery capability in multi-step task solving. Keywords: closed-loop language agent; multi-step task solving; step verification; local correction; process supervision; large language models · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On mathematical reasoning tasks, the proposed framework achieves an accuracy of 0.650, compared with 0.317 for Integrated Training, 0.460 for CoT Training, 0.568 for ReFT, and 0.617 for Agent Lumos.”
- Also stated in: § Abstract Multi-step task solving with large language models in intelligent electronic systems and interactive environments requires stronger process control and execution reliability. To address local error accumulation during multi-step execution, this paper proposes a closedloop modular language agent framework integrating task planning, action execution, step verification, local regeneration, and replanning. A process-supervision data construction method is further introduced, in which real execution steps are retained as valid samples and invalid samples are automatically synthesized through action substitution, input perturbation, observation replacement, and subgoal mismatch, providing step-level supervision for validity prediction. In the proposed framework, step verification functions as a process-level control signal that supports hierarchical recovery through local regeneration and replanning, rather than as a standalone filtering module. Experiments are conducted on mathematical reasoning and web interaction tasks. On mathematical reasoning tasks, the proposed framework achieves an accuracy of 0.650, compared with 0.317 for Integrated Training, 0.460 for CoT Training, 0.568 for ReFT, and 0.617 for Agent Lumos. On web interaction tasks, the proposed framework achieves a step success rate of 0.424, compared with 0.246 for Integrated Training and 0.310 for Agent Lumos. Among the cases where recovery is triggered, local regrounding succeeds in 85.7% of reground attempts, while replanning succeeds in 53.3% of replanning attempts. These results indicate that the proposed framework improves process stability and recovery capability in multi-step task solving. Keywords: closed-loop language agent; multi-step task solving; step verification; local correction; process supervision; large language models · plain
“On mathematical reasoning tasks, the proposed framework achieves an accuracy of 0.650”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Abstract Multi-step task solving with large language models in intelligent electronic systems and interactive environments requires stronger process control and execution reliability. To address local error accumulation during multi-step execution, this paper proposes a closedloop modular language agent framework integrating task planning, action execution, step verification, local regeneration, and replanning. A process-supervision data construction method is further introduced, in which real execution steps are retained as valid samples and invalid samples are automatically synthesized through action substitution, input perturbation, observation replacement, and subgoal mismatch, providing step-level supervision for validity prediction. In the proposed framework, step verification functions as a process-level control signal that supports hierarchical recovery through local regeneration and replanning, rather than as a standalone filtering module. Experiments are conducted on mathematical reasoning and web interaction tasks. On mathematical reasoning tasks, the proposed framework achieves an accuracy of 0.650, compared with 0.317 for Integrated Training, 0.460 for CoT Training, 0.568 for ReFT, and 0.617 for Agent Lumos. On web interaction tasks, the proposed framework achieves a step success rate of 0.424, compared with 0.246 for Integrated Training and 0.310 for Agent Lumos. Among the cases where recovery is triggered, local regrounding succeeds in 85.7% of reground attempts, while replanning succeeds in 53.3% of replanning attempts. These results indicate that the proposed framework improves process stability and recovery capability in multi-step task solving. Keywords: closed-loop language agent; multi-step task solving; step verification; local correction; process supervision; large language models · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“0.310 for Agent Lumos”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
60 more claims
Create a free account to see all 158 claims and to verify the 9 references still open.
- Probably not a substantial claim. 37
- It may state two separate claims as one. 62
- The quote only partly states this claim. 3
Citations
Cites
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Self-Refine: Iterative Refinement with Self-Feedback
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.17651 · Cited by 2,314
Toolformer: Language Models Can Teach Themselves to Use Tools
Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023) · arXiv · doi:10.48550/arxiv.2302.04761 · Cited by 1,150
Reflexion: Language Agents with Verbal Reinforcement Learning
Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11366 · Cited by 1,087
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language ModelsPreprint
Wang, L., Xu, W., Lan, Y.-H., Hu, Z.-Q., Lan, Y.-S., Lee, R. K.-W., & Lim, E.-P. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2305.04091 · Cited by 946
Training Verifiers to Solve Math Word Problems
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.14168 · Cited by 575
CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing
Gou, Z., Shao, Z., Gong, Y., Shen, Y., Yang, Y., Duan, N., & Chen, W. (2023) · arXiv · doi:10.48550/arxiv.2305.11738 · Cited by 561
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models
Wang, L., Xu, W., Lan, Y., Hu, Z., Lan, Y., Lee, R. K.-W., & Lim, E.-P. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.147 · Cited by 344
SELF-DISCOVER: Large Language Models Self-Compose Reasoning Structures
Zhou, P., Pujara, J., Ren, X., Chen, X., Cheng, H.-T., Le, Q. V., Ed, H., Zhou, D., Mishra, S. R., & Zheng, H. (2024) · neural information processing systems · doi:10.52202/079017-4004 · Cited by 224
AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planning
Qiao, S., Zhang, N., Fang, R., Luo, Y., Zhou, W., Jiang, Y., Lv, C., & Chen, H. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.165 · Cited by 179
Research works printed but not linked: 1
- 4. Yao, S.; Zhao, J.; Yu, D.; Du, N.; Shafran, I.; Narasimhan, K.R.; Cao, Y. ReAct: Synergizing Reasoning and Acting in Language Models. In Proceedings of the Eleventh International Conference on Learning Representations (ICLR 2023), Kigali, Rwanda, 1-5 May 2023.
Cited by
Component Replacement in LLM Agents: A Critical Scoping Review of Task Benefit and AttributionPreprint
Zhang, S., & Chang, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.39989
Metadata
- Authors
- Li, H., Feng, L., & Wang, D.
- Affiliations
- School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China
- Nanjing Tech University (CN)
- Southeast University (CN)
- Corresponding author
- Wang, Dong
- Venue
- Electronics
- Publisher
- MDPI AG
- Published
- 2026-05-09
- Volume
- 15
- Issue
- 10
- Pages
- 2011
- ISSN
- 2079-9292 (ISSN-L)
- OpenAlex
- W7160826567
- Semantic Scholar
- 57ab7a0a908a32a79933d494ec49ef9a63522221
- Cited by
- 0
- References
- 13
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Electronics (publishedVersion, cc-by)
- Funders
- National Natural Science Foundation of China (62103184)
- Jiangsu Science and Technology Programme (BZ2024057)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- large language models, local regeneration, replanning, process supervision, mathematical reasoning, web interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.