Agents4plc: Automating closed-loop plc code generation and verification in industrial control systems using llm-based agents
Open accessAbstract
In industrial control systems, the generation and verification of Programmable Logic Controller (PLC) code are crucial for ensuring operational efficiency and safety. While Large Language Models (LLMs) have made strides in automated code generation, they fall short in providing correctness guarantees and specialized support for PLC programming (which has its own programming language and clear logical structures). To address these challenges, this paper introduces Agents4PLC, a novel framework that not only automates PLC code generation but also introduces code-level verification and repair built upon an LLM-based multi-agent system, which together is capable of directly producing operational PLC code without any human interaction. To comprehensively evaluate our framework, we first establish a new benchmark specially designed for the critical area ofverifiable PLC code generation, which includes hundreds of natural language requirements, human-written and verified formal specifications, and finally reference PLC code. Then, we carefully designed a multi-agent workflow combining a set of expert agents responsible for different code generation tasks including planning, coding, validation and debugging towards generating correct PLC code. For each agent, we also incorporate optimization strategies such as Retrieval-Augmented Generation (RAG), advanced prompt engineering techniques, and Chain-of-Thought strategies which are shown to be effective to enhance the ability of these expert ‘agents’. Evaluation against the benchmark demonstrates that Agents4PLC significantly outperforms existing methods, achieving superior results across a series of increasingly rigorous evaluation metrics. This research highlights the potential of LLM agent-based code generation in real-world industrial control systems and the importance of code-level verification in generating correct code with formal guarantees.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Verification-driven tool use uses tests, execution results, compiler errors, runtime traces, type checkers, static analyzers, and verifier feedback as the main signals for improving code quality.
“Instead, they use tests, execution results, compiler errors, runtime traces, type checkers, static analyzers, and verifier feedback as the main signals for improving code quality [226, 235, 236, 237].”
From Code as Agent Harness (2026) · cites this paper as [235]
The Debugging Agent and Optimization sections (pages not provided) describe using compiler results, error messages, test names, and formal-verification outcomes to repair generated PLC code. The paper does not establish the full list of signals in the citing sentence: our copy does not describe runtime traces, type checkers, or static analyzers as feedback used by this system. (LLM, conf=0.87)
Experimental — accuracy not yet measuredCode supports governance mechanisms for industrial safety and verified closed-loop control.
“For auditability, it supports governance mechanisms for industrial safety [119, 235] and verified closed-loop control [115].”
From Code as Agent Harness (2026) · cites this paper as [235]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941 · see citing works
Competition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043 · see citing works
ChatDev: Communicative Agents for Software Development
Qian, C., Liu, W., Liu, H., Chen, N., Dang, Y., Li, J., Yang, C., Chen, W., Su, Y., Cong, X., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.810 · Cited by 490 · see citing works
Self-citationA Survey on the Memory Mechanism of Large Language Model-based Agents
Zhang, Z., Dai, Q., Bo, X., Ma, C., Li, R., Chen, X., Zhu, J., Dong, Z., & Wen, J.-R. (2025) · ACM Transactions on Information Systems · doi:10.1145/3748302 · Cited by 118 · see citing works
Demystifying LLM-Based Software Engineering Agents
Xia, C. S., Deng, Y., Dunn, S., & Zhang, L. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3715754 · Cited by 87 · see citing works
MapCoder: Multi-Agent Code Generation for Competitive Problem Solving
Islam, M. A., Ali, M. E., & Parvez, M. R. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.269 · Cited by 63 · see citing works
Flagged first, then by citations
Research works printed but not linked: 43
- D. Bohlender, H. Simon, N. Friedrich, S. Kowalewski, and S. Hauck- Stattelmann, "Concolic test generation for plc programs using coverage metrics," in 2016 13th International Workshop on Discrete Event Sys- tems (WODES). IEEE, 2016, pp. 432-437.
- R. Cavada, A. Cimatti, M. Dorigatti, A. Griggio, A. Mariotti, A. Micheli, S. Mover, M. Roveri, and S. Tonetta, "The nuxmv symbolic model checker," in Computer Aided Verification: 26th International Confer- ence, CAV 2014, Held as Part of the Vienna Summer of Logic, VSL 2014, Vienna, Austria, July 18-22, 2014. Proceedings 26. Springer, 2014, pp. 334-342.
- M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman et al., "Evaluating large language models trained on code," arXiv preprint arXiv:2107.03374, 2021. · arXiv
- Z. Chen, S. Fang, and M. Monperrus, "Supersonic: Learning to generate source code optimizations in c/c++," IEEE Transactions on Software Engineering, 2024.
- V. Corso, L. Mariani, D. Micucci, and O. Riganelli, "Generating java methods: An empirical assessment of four ai-based code assistants," in Proceedings of the 32nd IEEE/ACM International Conference on Program Comprehension, 2024, pp. 13-23.
- D. Darvas, E. Blanco Vinuela, and B. Fernández Adiego, "Plcverif: A tool to verify plc programs based on model checking techniques," in Proceedings of the 15th International Conference on Accelerator and Large Experimental Physics Control Systems. IEEE, 2015, pp. 1-6.
- M. Ebrahimi Salari, E. P. Enoiu, W. Afzal, and C. Seceleanu, "Pylc: A framework for transforming and validating plc software using python and pynguin test generator," in Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing, 2023, pp. 1476-1485.
- M. Fakih, R. Dharmaji, Y. Moghaddas, G. Quiros, O. Ogundare, and M. A. Al Faruque, "Llm4plc: Harnessing large language models for verifiable programming of plcs in industrial control systems," in Pro- ceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice, 2024, pp. 192-203.
- E. First and Y. Brun, "Diversity-driven automated formal verification," in Proceedings of the 44th International Conference on Software Engi- neering, 2022, pp. 749-761.
- D. Guo, Q. Zhu, D. Yang, Z. Xie, K. Dong, W. Zhang, G. Chen, X. Bi, Y. Wu, Y. Li et al., "Deepseek-coder: When the large language model meets programming-the rise of code intelligence," arXiv preprint arXiv:2401.14196, 2024. · arXiv
- S. Hong, X. Zheng, J. Chen, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou et al., "Metagpt: Meta programming for multi-agent collaborative framework," arXiv preprint arXiv:2308.00352, 2023. · arXiv
- X. Hou, Y. Zhao, Y. Liu, Z. Yang, K. Wang, L. Li, X. Luo, D. Lo, J. Grundy, and H. Wang, "Large language models for software engi- neering: A systematic literature review," ACM Transactions on Software Engineering and Methodology, 2023.
- D. Huang, Q. Bu, J. M. Zhang, M. Luck, and H. Cui, "Agentcoder: Multi-agent-based code generation with iterative testing and optimisa- tion," arXiv preprint arXiv:2312.13010, 2023. · arXiv
- International Electrotechnical Commission (IEC), "IEC 61131- 3:2013 Programmable controllers -Part 3: Programming languages," https://webstore.iec.ch/en/publication/4552, 2013, edition 3.0, ISBN: 9782832206614.
- C. Isbell, C. R. Shelton, M. Kearns, S. Singh, and P. Stone, "A social reinforcement learning agent," in Proceedings of the fifth international conference on Autonomous agents, 2001, pp. 377-384.
- L. P. Kaelbling, M. L. Littman, and A. W. Moore, "Reinforcement learning: A survey," Journal of artificial intelligence research, vol. 4, pp. 237-285, 1996.
- M. Kornaszewski, "The use of programmable logic controllers in railway signaling systems," in ICTE in Transportation and Logistics 2019. Springer, 2020, pp. 104-111.
- H. Koziolek, V. Ashiwal, S. Bandyopadhyay et al., "Automated control logic test case generation using large language models," arXiv preprint arXiv:2405.01874, 2024. · arXiv
- H. Koziolek, S. Gruener, and V. Ashiwal, "Chatgpt for plc/dcs control logic generation," in 2023 IEEE 28th International Conference on Emerging Technologies and Factory Automation (ETFA). IEEE, 2023, pp. 1-8.
- H. Koziolek, S. Grüner, R. Hark, V. Ashiwal, S. Linsbauer, and N. Es- kandani, "Llm-based and retrieval-augmented control code generation," in Proceedings of the 1st International Workshop on Large Language Models for Code, 2024, pp. 22-29.
- H. Koziolek and A. Koziolek, "Llm-based control code generation using image recognition," in Proceedings of the 1st International Workshop on Large Language Models for Code, 2024, pp. 38-45.
- D. Kroening and M. Tautschnig, "Cbmc-c bounded model checker: (competition contribution)," in Tools and Algorithms for the Construc- tion and Analysis of Systems: 20th International Conference, TACAS 2014, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2014, Grenoble, France, April 5-13, 2014. Proceedings 20. Springer, 2014, pp. 389-391.
- J. Lee and K. Bae, "Formal semantics and analysis of multitask plc st programs with preemption," in International Symposium on Formal Methods. Springer, 2024, pp. 425-442.
- J. Lee, S. Kim, and K. Bae, "Bounded model checking of plc st programs using rewriting modulo smt," in Proceedings of the 8th ACM SIGPLAN International Workshop on Formal Techniques for Safety- Critical Systems, 2022, pp. 56-67.
- R. Li, L. B. Allal, Y. Zi, N. Muennighoff, D. Kocetkov, C. Mou, M. Marone, C. Akiki, J. Li, J. Chim et al., "Starcoder: may the source be with you!" arXiv preprint arXiv:2305.06161, 2023. · arXiv
- J. Liu, K. Wang, Y. Chen, X. Peng, Z. Chen, L. Zhang, and Y. Lou, "Large language model-based agents for software engineering: A sur- vey," arXiv preprint arXiv:2409.02977, 2024. · arXiv
- M. Minsky, "Steps toward artificial intelligence," Proceedings of the IRE, vol. 49, no. 1, pp. 8-30, 1961.
- A. Nunez, N. T. Islam, S. K. Jha, and P. Najafirad, "Autosafecoder: A multi-agent framework for securing llm code generation through static analysis and fuzz testing," arXiv preprint arXiv:2409.10737, 2024. · arXiv
- T. Ovatman, A. Aral, D. Polat, and A. O. Ünver, "An overview of model checking practices on verification of plc software," Software & Systems Modeling, vol. 15, no. 4, pp. 937-960, 2016.
- C. Qian, W. Liu, H. Liu, N. Chen, Y. Dang, J. Li, C. Yang, W. Chen, Y. Su, X. Cong, J. Xu, D. Li, Z. Liu, and M. Sun, "Chatdev: Communicative agents for software development," arXiv preprint arXiv:2307.07924, 2023. [Online]. Available: https://arxiv.org/ abs/2307.07924 · arXiv
- C. Ribeiro, "Reinforcement learning agents," Artificial intelligence re- view, vol. 17, pp. 223-250, 2002.
- B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y. Adi, J. Liu, R. Sauvestre, T. Remez et al., "Code llama: Open foundation models for code," arXiv preprint arXiv:2308.12950, 2023. · arXiv
- M. Schreyer and M. M. Tseng, "Design framework of plc-based control for reconfigurable manufacturing systems," in Proceedings of interna- tional conference on flexible automation and intelligent manufacturing (FAIM 2000), vol. 1, 2000, pp. 33-42.
- J. Shi, Y. Chen, Q. Li, Y. Huang, Y. Yang, and M. Zhao, "Automated test cases generator for iec 61131-3 structured text based dynamic symbolic execution," IEEE Transactions on Computers, 2024.
- M. L. Siddiq, B. Casey, and J. Santos, "A lightweight framework for high-quality code generation," arXiv preprint arXiv:2307.08220, 2023. · arXiv
- A. Singh, "Taxonomy of machine learning techniques in test case gen- eration," in 2023 7th International Conference on Intelligent Computing and Control Systems (ICICCS). IEEE, 2023, pp. 474-481.
- D. Tang, Z. Chen, K. Kim, Y. Song, H. Tian, S. Ezzini, Y. Huang, and J. K. T. F. Bissyande, "Collaborative agents for software engineering," arXiv preprint arXiv:2402.02172, 2024. · arXiv
- K. Wang, J. Wang, C. M. Poskitt, X. Chen, J. Sun, and P. Cheng, "K-st: A formal executable semantics of the structured text language for plcs," IEEE Transactions on Software Engineering, 2023.
- M. Wang, "Application of plc technology in electrical engineering and automation control," in Application of Intelligent Systems in Multi- modal Information Analytics: Proceedings of the 2020 International Conference on Multi-model Information Analytics (MMIA2020), Volume 2. Springer, 2021, pp. 131-135.
- Z. Xi, W. Chen, X. Guo, W. He, Y. Ding, B. Hong, M. Zhang, J. Wang, S. Jin, E. Zhou et al., "The rise and potential of large language model based agents: A survey," arXiv preprint arXiv:2309.07864, 2023. · arXiv
- C. S. Xia and L. Zhang, "Keep the conversation going: Fixing 162 out of 337 bugs for $0.42 each using chatgpt," arXiv preprint arXiv:2304.00385, 2023. · arXiv
- K. Xu, G. L. Zhang, X. Yin, C. Zhuo, U. Schlichtmann, and B. Li, "Automated c/c++ program repair for high-level synthesis via large language models," in Proceedings of the 2024 ACM/IEEE International Symposium on Machine Learning for CAD, 2024, pp. 1-9.
- K. Zhang, J. Li, G. Li, X. Shi, and Z. Jin, "Codeagent: Enhancing code generation with tool-integrated agent systems for real-world repo-level coding challenges," arXiv preprint arXiv:2401.07339, 2024. · arXiv
Cited by
Sorted by citations
Ning, X., Tieu, K., Fu, D., Wei, T., Li, Z., Bei, Y., Zou, J., Ai, M., Liu, Z., Li, T.-W., et al. (2026) · arXiv · doi:10.48550/arxiv.2605.18747
Self-citation
- Authors
- Liu, Z., Zeng, R., Wang, D., Peng, G., Liu, X., Liu, Q., Liu, P., Wang, W., & Wang, J.
- Affiliations
- Zhejiang University (CN)
- UWin Tech, Hangzhou, China
- ORCID
- Venue
- IEEE Transactions on Software Engineering
- Published
- 2026-05
- Volume
- 52
- Issue
- 5
- Pages
- 1672-1687
- ISSN
- 0098-5589 (ISSN-L)
- 1939-3520
- 2326-3881
- OpenAlex
- W7131417432
- Cited by
- 20
- References
- 6
- Citations by year
- 2026: 17 · 2025: 3
- FWCI
- 144.61
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Software Engineering (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Engineering › Control and Systems Engineering › Industrial Automation and Control Systems
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- PLC code generation, large language models, multi-agent systems, code verification, code repair, industrial control systems, retrieval-augmented generation, chain-of-thought prompting, prompt engineering, formal specification, benchmark, automatic code generation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
