CoChat
Sign Up
PreprintCited by 1Topic ModelingMathematics, Computing, and Information ProcessingNatural Language Processing Techniques

NeuroProlog: Multi-Task Fine-Tuning for Neurosymbolic Mathematical Reasoning via the Cocktail Effect

Open access
Zunjare, Pratibha, Hsiao, Michael·2026·arXiv
DOI 10.48550/arxiv.2603.02504arXiv 2603.02504
Open this paper in CoChat — free accountOpen PDF

Abstract

Large Language Models (LLMs) achieve strong performance on natural language tasks but remain unreliable in mathematical reasoning, frequently generating fluent yet logically inconsistent solutions. We present \textbf{NeuroProlog}, a neurosymbolic framework that ensures verifiable reasoning by compiling math word problems into executable Prolog programs with formal verification guarantees. We propose a multi-task Cocktail training strategy that jointly optimizes three synergistic objectives in a unified symbolic representation space: (i) mathematical formula-to-rule translation (KB), (ii) natural language-to-program synthesis (SOLVE), and (iii) program-answer alignment. This joint supervision enables positive transfer, where symbolic grounding in formula translation directly improves compositional reasoning capabilities. At inference, we introduce an execution-guided decoding pipeline with fine-grained error taxonomy that enables iterative program repair and quantifies model self-debugging capacity. Evaluation on GSM8K across multiple model scales demonstrates that cocktail training improves accuracy over single-task baselines, with statistically significant gains for most evaluated models. Error analysis reveals scale-associated differences in repair behavior: larger models exhibit more readily correctable errors, whereas smaller models show reduced syntactic errors but persistent semantic failures. These findings suggest that model capacity influences the acquisition of reliable symbolic reasoning and self-correction capabilities.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2603.02504

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 39

  1. Meni Brief, Oded Ovadia, Gil Shenderovitz, Noga Ben Yoash, Rachel Lemberg, and Eitam Sheetrit. 2024. Mixing It Up: The Cocktail Effect of Multi-Task Fine-Tuning on LLM Performance -A Case Study in Finance. doi:10.48550/arXiv.2410.01109 · DOI
  2. Rich Caruana. 1997. Multitask Learning. Machine Learning 28, 1 (1997), 41-75. doi:10.1023/A:1007379606734 · DOI · Paper page
  3. Mark Chen, Jerry Tworek, Heewoo Jun, and et al. 2021. Evaluating Large Lan- guage Models Trained on Code. CoRR abs/2107.03374 (2021). arXiv:2107.03374 https://arxiv.org/abs/2107.03374 · arXiv
  4. Michael K. Chen. 2025. A Comparative Study of Neurosymbolic AI Approaches to Interpretable Logical Reasoning. In Proceedings of NeSy 2025.
  5. Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. 2022. Pro- gram of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks. doi:10.48550/arXiv.2211.12588 · DOI · Paper page
  6. Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. 2024. Teach- ing large language models to self-debug. International Conference on Learning Representations (ICLR) (2024).
  7. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training Verifiers to Solve Math Word Problems. arXiv preprint arXiv:2110.14168 (2021). · arXiv
  8. Antonia Creswell and Murray Shanahan. 2022. Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning. arXiv preprint arXiv:2205.09712 (2022). https://arxiv.org/abs/2205.09712 · arXiv
  9. Andrew Cropper and Rolf Morel. 2021. Learning programs by learning from failures. Machine Learning 110 (2021), 801-856. doi:10.1007/s10994-020-05934-z · DOI · Paper page
  10. Sunhao Dai, Weihao Liu, Yuqi Zhou, Liang Pang, Rongju Ruan, Gang Wang, Zhenhua Dong, Jun Xu, and Ji-Rong Wen. 2024. Cocktail: A Comprehensive Information Retrieval Benchmark with LLM-Generated Documents Integration. In Findings of the Association for Computational Linguistics: ACL 2024. Association for Computational Linguistics, 7052-7074. doi:10.18653/v1/2024.findings-acl.421 · DOI
  11. Thomas G Dietterich. 1998. Approximate statistical tests for comparing su- pervised classification learning algorithms. Neural Computation 10, 7 (1998), 1895-1923.
  12. Nouha Dziri, Ximing Lu, Melanie Sclar, Yejin Choi, and Ari Holtzman. 2023. Faithful Chain-of-Thought Reasoning. arXiv preprint arXiv:2301.13379 (2023). · arXiv
  13. Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. PAL: Program-aided language models. 202 (2023), 10764-10799. https://proceedings.mlr.press/v202/gao23f.html
  14. Artur d'Avila Garcez, Marco Gori, Luis C Lamb, Luciano Serafini, Michael Spranger, and Son N Tran. 2019. Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning. Jour- nal of Applied Logics 6, 4 (2019), 611-632.
  15. Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Minlie Huang, Nan Duan, and Weizhu Chen. 2023. ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving. arXiv:2309.17452 [cs.CL] https://arxiv.org/ abs/2309.17452 · arXiv
  16. Henry Kautz. 2022. The third AI summer: AAAI Robert S. Engelmore memorial lecture. AI Magazine 43, 1 (2022), 105-125.
  17. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large Language Models are Zero-Shot Reasoners. doi:10.48550/ arXiv.2205.11916 · DOI · Paper page
  18. Aitor Lewkowycz, Anders Andreassen, David Dohan, et al. 2022. Solving Quantitative Reasoning Problems with Language Models. arXiv preprint arXiv:2206.14858 (2022). · arXiv
  19. Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. 2022. Competition-level code generation with AlphaCode. Science 378, 6624 (2022), 1092-1097. https://www.science.org/doi/10.1126/science.abq1158 · DOI · Paper page
  20. Yuan Li, Zhengzhong Liu, and Eric Xing. 2025. Data mixing optimization for supervised fine-tuning of large language models. In Proceedings of the 42nd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 267). PMLR, 35419-35437. https://proceedings.mlr.press/v267/ li25bh.html
  21. Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2023. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747 (2023). https://arxiv.org/ abs/2308.08747 · arXiv
  22. Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Mari- anna Apidianaki, and Chris Callison-Burch. 2023. Faithful Chain-of-Thought Reasoning. (Nov. 2023), 305-329. doi:10.18653/v1/2023.ijcnlp-main.20 · DOI · Paper page
  23. Xuefei Ning, Zifu Wang, Shiyao Li, Zinan Lin, Peiran Yao, Tianyu Fu, Matthew B. Blaschko, Guohao Dai, Huazhong Yang, and Yu Wang. 2024. Can LLMs Learn by Teaching for Better Reasoning? A Preliminary Study. In Advances in Neu- ral Information Processing Systems 37. Neural Information Processing Systems Foundation, Inc., 71188-71239. doi:10.52202/079017-2275 · DOI
  24. Theo X. Olausson, Alex Gu, Benjamin Lipkin, Cedegao E. Zhang, Armando Solar- Lezama, Joshua B. Tenenbaum, and Roger Levy. 2023. LINC: A neurosymbolic approach for logical reasoning by combining language models with first-order logic provers. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 5153- 5176. doi:10.18653/v1/2023.emnlp-main.313 · DOI · Paper page
  25. Liangming Pan, Alon Albalak, Xinyi Wang, and William Wang. 2023. Logic- LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2023. Association for Computational Linguistics, Singapore, 3806-3824. doi:10.18653/v1/2023.findings-emnlp.248 · DOI · Paper page
  26. Liangming Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. 2023. Logic- LM: Empowering large language models with symbolic solvers for faithful logical reasoning. Findings of the Association for Computational Linguistics: EMNLP (2023).
  27. Yasaman Razeghi, Robert L. Logan IV, Matt Gardner, and Sameer Singh. 2022. Impact of pretraining term frequencies on few-shot numerical reasoning. (2022), 840-854. doi:10.18653/v1/2022.findings-emnlp.59 · DOI · Paper page
  28. Baptiste Rozière et al. 2023. Code Llama: Open Foundation Models for Code. arXiv preprint arXiv:2308.12950 (2023). · arXiv
  29. Sebastian Ruder. 2017. An Overview of Multi-Task Learning in Deep Neural Networks. arXiv preprint arXiv:1706.05098 (2017). https://arxiv.org/abs/1706. 05098 · arXiv
  30. Victor Sanh, Albert Webson, Colin Raffel, and et al. 2021. Multitask prompted training enables zero-shot task generalization. arXiv preprint arXiv:2110.08207 (2021). https://arxiv.org/abs/2110.08207 · arXiv
  31. Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language Agents with Verbal Re- inforcement Learning. In Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS 2023). https://arxiv.org/abs/2303.11366 · arXiv
  32. Vikash Singh, Darion Cassel, Nathaniel Weir, et al. 2026. VERGE: Formal Refine- ment and Guidance Engine for Verifiable LLM Reasoning. (2026). arXiv preprint arXiv:2601.20055. · arXiv
  33. Shubham Toshniwal, Ivan Moshkov, Sean Narenthiran, Daria Gitman, Fei Jia, and Igor Gitman. 2024. OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset. arXiv preprint arXiv:2402.10176 (2024). https://arxiv.org/abs/2402.10176 · arXiv
  34. Karthik Valmeekam, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambham- pati. 2022. Large language models still can't plan (a benchmark for LLMs on planning and reasoning about change). arXiv preprint arXiv:2206.10498 (2022). https://arxiv.org/abs/2206.10498 · arXiv
  35. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain-of-thought reasoning in language models. arXiv preprint arXiv:2203.11171 (2022). https://arxiv.org/abs/2203.11171 · arXiv
  36. Yue Wang, Hung Le, Akhilesh Deepak Gotmare, Nghi D. Q. Bui, Junnan Li, and Steven C. H. Hoi. 2023. CodeT5+: Open code large language models for code understanding and generation. (2023), 1069-1088. doi:10.18653/v1/2023.emnlp- main.68 · DOI · Paper page
  37. Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2022. Finetuned language models are zero-shot learners. International Conference on Learning Representations (ICLR) (2022).
  38. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903 (2022). https://arxiv.org/abs/2201.11903 · arXiv
  39. Xiaocheng Yang, Bingsen Chen, and Yik-Cheung Tam. 2024. Arithmetic Reason- ing with LLM: Prolog Generation & Permutation. In Proceedings of the 2024 Con- ference of the North American Chapter of the Association for Computational Linguis- tics: Human Language Technologies (Volume 2: Short Papers). Association for Com- putational Linguistics, Mexico City, Mexico, 699-710. doi:10.18653/v1/2024.naacl- short.61 · DOI · Paper page

Cited by

Sorted by citations

Authors
Zunjare, P., & Hsiao, M.
Affiliations
Virginia Tech (US)
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2603.02504
arXiv
2603.02504
OpenAlex
W7133502127
Cited by
0
References
1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Mathematics, Computing, and Information Processing
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
neuro-symbolic reasoning, mathematical reasoning, Prolog, mathematical word problems, multi-task learning, program synthesis, error analysis, self-correction, model capacity, GSM8K

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review