CoChat
Sign Up
Cited by 25Web Application Security VulnerabilitiesAdversarial Robustness in Machine LearningNetwork Security and Intrusion Detection

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

PenHeal: A Two-Stage LLM Framework for Automated Pentesting and Optimal Remediation

Open access
Huang, Junjie, Zhu, Quanyan·2023·Proceedings of the Workshop on Autonomous Cybersecurity
DOI 10.1145/3689933.3690831
Open this paper in CoChat — free accountOpen PDF

Abstract

Recent advances in Large Language Models (LLMs) have shown significant potential in enhancing cybersecurity defenses against sophisticated threats. LLM-based penetration testing is an essential step in automating system security evaluations by identifying vulnerabilities. Remediation, the subsequent crucial step, addresses these discovered vulnerabilities. Since details about vulnerabilities, exploitation methods, and software versions offer crucial insights into system weaknesses, integrating penetration testing with vulnerability remediation into a cohesive system has become both intuitive and necessary.

Overview

Integrity signals

Registry

Crossref · 10.1145/3689933.3690831

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • ReAct, plan-and-solve, and reflection are the most influential control loops for agentic AI.

    “These capabilities are coordinated by control loops, the most influential of which are ReAct (Yao S. et al., ICLR 2023 [89]), which interleaves natural-language reasoning with tool invocations [25]; plan-and-solve, which produces a complete plan before execution; and reflection, which critiques prior outputs before committing [28,32].”

    From Agentic AI and Large Language Models for Autonomous IoT Cybersecurity: A Systematic Survey, Taxonomy, and Research Roadmap (2026) · cites this paper as [32]

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 36

  1. OpenAI et al. 2024. Gpt-4 technical report. (2024). arXiv: 2303.08774. · arXiv
  2. Hugo Touvron et al. 2023. Llama 2: open foundation and fine-tuned chat models. (2023). arXiv: 2307.09288. · arXiv
  3. Gemini Team et al. 2024. Gemini: a family of highly capable multimodal models. (2024). arXiv: 2312.11805. · arXiv
  4. 10.1145/3649449 · DOI
  5. QiangWang, Bei Li, Tong Xiao, Jingbo Zhu, Changliang Li, Derek F.Wong, and Lidia S. Chao. 2019. Learning deep transformer models for machine translation. (2019). https://arxiv.org/abs/1906.01787 arXiv: 1906.01787 [cs.CL]. · arXiv
  6. Haoyu Zhang, Jianjun Xu, and Ji Wang. 2019. Pretraining-based natural language generation for text summarization. (2019). https://arxiv.org/abs/1902.09 243 arXiv: 1902.09243 [cs.CL]. · arXiv
  7. 10.1109/W-FiCloud.2016.29 · DOI
  8. 10.1109/MSP.2007.159 · DOI
  9. Xin Zhou Sicong Cao Xiaobing Sun and David Lo. 2024. Large language model for vulnerability detection and repair: literature review and the road ahead. (2024). arXiv: 2404.02525. · arXiv
  10. Samaneh Shafee, Alysson Bessani, and Pedro M. Ferreira. 2024. Evaluation of llm chatbots for osint-based cyber threat awareness. (2024). arXiv: 2401.15127. · arXiv
  11. Marwan Omar. 2023. Detecting software vulnerabilities using language models. (2023). arXiv: 2302.11773. · DOI
  12. Gelei Deng et al. 2023. Pentestgpt: an llm-empowered automatic penetration testing tool. (2023). arXiv: 2308.06782 [cs.SE]. · arXiv
  13. Jiacen Xu Jack W. Stokes Geoff McDonald Xuesong Bai David Marshall Siyue Wang Adith Swaminathan and Zhou Li. 2024. Autoattacker: a large language model guided system to implement automatic cyber-attacks. (2024). arXiv: 2403.01038. · arXiv
  14. Georgia Weidman. 2014. Penetration Testing: A Hands-On Introduction to Hacking. No Starch Press, 528. isbn: 9781593275648.
  15. 10.1109/EuroSPW51379.2020.00010 · DOI
  16. 10.1109/HPCC/SmartCity/DSS.2018.00244 · DOI
  17. 10.1109/CNS48642.2020.9162301 · DOI
  18. 10.1145/3540250.3549098 · DOI
  19. 10.1145/3387940.3392200 · DOI
  20. 10.1109/ISSRE52982.2021.00031 · DOI
  21. 10.1109/tse.2022.3147265 · DOI
  22. Jacob Harer, Onur Ozdemir, Tomo Lazovich, Christopher P. Reale, Rebecca L. Russell, Louis Y. Kim, and Peter Chin. 2018. Learning to repair software vulnerabilities with generative adversarial networks. (2018). arXiv: 1805.07475. · arXiv
  23. MITRE Corporation. 2021. Common Vulnerabilities and Exposures (CVE). Accessed: 2024-06--21. The MITRE Corporation. https://cve.mitre.org.
  24. 2021. CVSS v3.1 Specification Document. https://www.first.org/cvss/v3.1/spec ification-document. Accessed: 2024-06--21. FIRST, (2021).
  25. Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez Lukasz Kaiser and Illia Polosukhin. 2023. Attention is all you need. (2023). arXiv: 1706.03762. · arXiv
  26. 10.1109/ICASSP39728.2021.9413810 · DOI
  27. Aobo Kong Shiwan Zhao Hao Chen Qicheng Li Yong Qin Ruiqi Sun Xin Zhou Enzhi Wang and Xiaohang Dong. 2024. Better zero-shot reasoning with role-play prompting. (2024). arXiv: 2308.07702. · DOI
  28. Yujia Qin et al. 2023. Toolllm: facilitating large language models to master 16000 real-world apis. (2023). arXiv: 2307.16789. · arXiv
  29. Tamás Vörös, Sean Paul Bergeron, and Konstantin Berlin. 2023. Web content filtering through knowledge distillation of large language models. (2023). arXiv: 2305.05027. · arXiv
  30. Mehrdad Kaheh, Danial Khosh Kholgh, and Panos Kostakos. 2023. Cyber sentinel: exploring conversational agents in streamlining security tasks with gpt-4. (2023). https://arxiv.org/abs/2309.16422 arXiv: 2309.16422 [cs.CR]. · arXiv
  31. John Yang, Akshara Prabhakar, Shunyu Yao, Kexin Pei, and Karthik R Narasimhan. 2023. Language agents as hackers: evaluating cybersecurity skills with capture the flag. In Multi-Agent Security Workshop @ NeurIPS'23. https://openreview.n et/forum?id=KOZwk7BFc3.
  32. Wesley Tann, Yuancheng Liu, Jun Heng Sim, Choon Meng Seah, and Ee-Chien Chang. 2023. Using large language models for cybersecurity capture-the-flag challenges and certification questions. (2023). arXiv: 2308.10443. · arXiv
  33. Minghao Shao Boyuan Chen Sofija Jancheska Brendan Dolan-Gavitt Siddharth Garg Ramesh Karri and Muhammad Shafique. 2024. An empirical evaluation of llms for solving offensive security challenges. (2024). arXiv: 2402 .11814.
  34. 10.1145/3611643.3613083 · DOI · Paper page
  35. Andreas Happe Aaron Kaplan and Jürgen Cito. 2024. Llms as hackers: autonomous linux privilege escalation attacks. (2024). arXiv: 2310.11409. · arXiv
  36. Nipun Jaswal, Daniel Teixeira, Abhinav Singh, and Monika Agarwal. 2018. Metasploit Penetration Testing Cookbook: Evade Antiviruses, Bypass Firewalls, and Exploit Complex Environments with the Most Widely Used Penetration Testing Framework. (3rd ed.). Packt Publishing, (Feb. 2018). isbn: 9781788623179.

Cited by

Sorted by citations

Metadata

Authors
Huang, J., & Zhu, Q.
Affiliations
  • New York University Shanghai (CN)
  • New York University (US)
ORCID
Venue
Proceedings of the Workshop on Autonomous Cybersecurity
Publisher
ACM
Published
2023-11-06
Pages
11-22
DOI
10.1145/3689933.3690831
OpenAlex
W4404130344
Cited by
25
References
3
Citations by year
2026: 15 · 2025: 9 · 2024: 1
FWCI
6.43
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University)
Funders
National Science Foundation (ECCS-1847056 and BCS-2122060)
Topics
  • Physical Sciences › Computer Science › Information Systems › Web Application Security Vulnerabilities
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
Keywords
large language models, automated penetration testing, vulnerability remediation, vulnerability detection, cybersecurity, two-stage framework, software security evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review