CP-Agent: Agentic Constraint Programming
Open accessAbstract
The translation of natural language to formal constraint models requires expertise in the problem domain and modeling frameworks. To explore the effectiveness of agentic workflows, we propose CP-Agent, a Python coding agent that uses the ReAct framework with a persistent IPython kernel. We provide the relevant domain knowledge as a project prompt of under 50 lines. The algorithm works by iteratively executing code, observing the solver's feedback, and refining constraint models based on execution results. We evaluate CP-Agent on 101 constraint programming problems from CP-Bench. We made minor changes to the benchmark to address systematic ambiguities in the problem specifications and errors in the ground-truth models. On the clarified benchmark, CP-Agent achieves perfect accuracy on all 101 problems. Our experiments show that minimal guidance outperforms detailed procedural scaffolding. Our experiments also show that explicit task management tools can have both positive and negative effects on focused modeling tasks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
As a safety boundary, code constrains admissible actions at execution time.
“As a safety boundary, it constrains admissible actions at execution time [119, 116, 226].”
From Code as Agent Harness (2026) · cites this paper as [116]
Related Work (p. 2) says the constraint solver guarantees the correctness of computations, while Architecture (p. 3) describes an agent executing Python code and receiving feedback. Neither passage says code acts as a safety boundary that restricts the agent’s admissible actions at execution time; searches for runtime safety and action restrictions found no such passage. (LLM, conf=0.73)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 16
- [6][6] Ian P. Gent and Toby Walsh. Csp lib : A benchmark library for constraints. In Joxan Jaffar, editor, Principles and Practice of Constraint Programming -CP'99, 5th International Conference, Alexandria, Virginia, USA, October 11-14, 1999, Proceedings , volume 1713 of Lecture Notes in Computer Science , pages 480-481. Springer, 1999. doi: 10.1007/978-3-540-48085-3\_36 . - · DOI
- [9][9] Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. LongLLMLingua: Accelerating and enhancing LLMs in long context scenarios via prompt compression. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024) , pages 1658-1677, 2024. URL: https: //aclanthology.org/2024.acl-long.91/ . -
- [10][10] Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. DSPy: Compiling declarative language model calls into state-of-the-art pipelines. In International Conference on Learning Representations (ICLR) , 2024. -
- [13][13] Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023 , 2023. URL: http://papers.nips.cc/paper\_files/paper/2023/ hash/43e9d647ccd3e4b7b5baab53f0368686-Abstract-Conference.html . -
- [14][14] Kostis Michailidis, Dimos Tsouros, and Tias Guns. CP-Bench: Evaluating large language models for constraint modelling. CoRR , abs/2506.06052, 2025. arXiv:2506.06052 , doi: 10.48550/ARXIV.2506.06052 . - · DOI
- [17][17] Rindranirina Ramamonjison, Timothy T. L. Yu, Raymond Li, Haley Li, Giuseppe Carenini, Bissan Ghaddar, Shiqi He, Mahdi Mostajabdaveh, Amin Banitalebi-Dehkordi, Zirui Zhou, and Yong Zhang. NL4Opt competition: Formulating optimization problems based on their natural language descriptions. In Marco Ciccone, Gustavo Stolovitzky, and Jacob Albrecht, editors, NeurIPS 2022 Competition Track, November 28 - December 9, 2022, Online , volume 220 of Proceedings of Machine Learning Research , pages 189-203. PMLR, 2022. URL: https: //proceedings.mlr.press/v220/ramamonjison22a.html . -
- [19][19] Timo Schick, Jane Dwivedi-Yu, Roberto Dess` ı, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 -16, 2023 , 2023. URL: http://papers.nips.cc/paper\_files/paper/2023/hash/ d842425e4bf79ba039352da0f658a906-Abstract-Conference.html . -
- [21][21] Stefan Szeider. Bridging Language Models and Symbolic Solvers via the Model Context Protocol. In Jeremias Berg and Jakob Nordstr¨ om, editors, 28th International Conference on Theory and Applications of Satisfiability Testing (SAT 2025) , volume 341 of Leibniz International Proceedings in Informatics (LIPIcs) , pages 30:1-30:12, Dagstuhl, Germany, 2025. Schloss Dagstuhl - Leibniz-Zentrum f¨ ur Informatik. URL: https://drops.dagstuhl. de/entities/document/10.4230/LIPIcs.SAT.2025.30 . - · DOI · Paper page
- [23][23] Stefan Szeider. ASP-Bench: From natural language to logic programs. In 2nd International Workshop on Neuro-Symbolic Software Engineering (NSE) , 2026. To appear at ICSE 2026. Preprint at https://arxiv.org/abs/2602.01171 . - · arXiv
- [24][24] Dimos Tsouros, H´ el` ene Verhaeghe, Serdar Kadıo˘ glu, and Tias Guns. Holy grail 2.0: From natural language to constraint models. CoRR , abs/2308.01589, 2023. URL: https://arxiv. org/abs/2308.01589 , arXiv:2308.01589 . - · arXiv
- [25][25] Nemika Tyagi, Mihir Parmar, Mohith Kulkarni, Aswin RRV, Nisarg Patel, Mutsumi Nakamura, Arindam Mitra, and Chitta Baral. Step-by-step reasoning to solve grid puzzles: Where do LLMs falter? In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, USA, November 12-16, 2024 , pages 19898-19915. Association for Computational Linguistics, 2024. doi:10.18653/V1/2024.EMNLP-MAIN.1111 . - · DOI
- [26][26] Mark Wallace. Practical applications of constraint programming. Constraints An Int. J. , 1(1/2):139-168, 1996. doi:10.1007/BF00143881 . - · DOI
- [28][28] Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable code actions elicit better LLM agents. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenReview.net, 2024. URL: https://openreview.net/forum?id=jJ9BoXAfFa . -
- [29][29] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022 , 2022. URL: http://papers.nips.cc/paper\_files/ paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html . -
- [31][31] John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. Swe-agent: Agent-computer interfaces enable automated software engineering. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang, editors, Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 -15, 2024 , 2024. URL: http://papers.nips.cc/paper\_files/paper/2024/hash/ 5a7c947568c1b1328ccc5230172e1e7c-Abstract-Conference.html . -
- [32][32] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 15, 2023 . OpenReview.net, 2023. URL: https://openreview.net/forum?id=WE\_vluYUL-X .
Cited by
Sorted by citations
Ning, X., Tieu, K., Fu, D., Wei, T., Li, Z., Bei, Y., Zou, J., Ai, M., Liu, Z., Li, T.-W., et al. (2026) · arXiv · doi:10.48550/arxiv.2605.18747
Proceedings 42nd International Conference on Logic Programming
Szeider, S. (2026) · Electronic Proceedings in Theoretical Computer Science · doi:10.4204/eptcs.450.10
Self-citation
- Authors
- Szeider, S.
- Affiliations
- TU Wien (AT)
- Corresponding author
- Szeider, Stefan
- ORCID
- Szeider, Stefan
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- Pages
- 6-13
- arXiv
- 2508.07468
- OpenAlex
- W4416242278
- Cited by
- 1
- References
- 0
- Citations by year
- 2026: 1
- FWCI
- 10.36
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International Workshop on Large Language Models for Code (LLM4Code) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Constraint Satisfaction and Optimization
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
- Keywords
- constraint programming, coding agents, ReAct framework, agentic workflows
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
