CoChat
Sign Up
Cited by 49Web Application Security VulnerabilitiesAdversarial Robustness in Machine LearningNetwork Security and Intrusion Detection

Generative AI for pentesting: The good, the bad, the ugly

Open access
Hilario, Eric, Azam, Sami, Sundaram, Jawahar, Imran Mohammed, Khwaja, Shanmugam, Bharanidharan·2024·International Journal of Information Security
DOI 10.1007/s10207-024-00835-x
Open this paper in CoChat — free account

Abstract

This paper examines the role of Generative AI (GenAI) and Large Language Models (LLMs) in penetration testing exploring the benefits, challenges, and risks associated with cyber security applications. Through the use of generative artificial intelligence, penetration testing becomes more creative, test environments are customised, and continuous learning and adaptation is achieved. We examined how GenAI (ChatGPT 3.5) helps penetration testers with options and suggestions during the five stages of penetration testing. The effectiveness of the GenAI tool was tested using a publicly available vulnerable machine from VulnHub. It was amazing how quickly they responded at each stage and provided better pentesting report. In this article, we discuss potential risks, unintended consequences, and uncontrolled AI development associated with pentesting.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s10207-024-00835-x

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Tools such as GitHub Copilot can accelerate coding by generating syntactically correct suggestions but are prone to introducing vulnerabilities, inefficiencies, and logic errors, especially in complex or unfamiliar contexts.

    “In programming education, tools such as GitHub Copilot can accelerate coding by generating syntactically correct suggestions; however, they are also prone to introducing vulnerabilities, inefficiencies, and logic errors -especially in complex or unfamiliar contexts (Hilario et al., 2024;Zhang et al., 2023).”

    From Integrating Generative AI into Programming Education: Student Perceptions and the Challenge of Correcting AI Errors (2025) · cites this paper as Hilario et al., 2024

    Section 2.3.1 (page unavailable) says generative AI can save time in penetration testing, and Section 2.4.1 (page unavailable) warns that AI results can contain false positives or negatives. Those points support the broad efficiency-and-error framing, but the paper does not establish the claim about GitHub Copilot’s coding suggestions, syntactic correctness, or their propensity to introduce vulnerabilities, inefficiencies, and logic errors in complex or unfamiliar programming contexts. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. [6]6. Regularly review and update user privileges, and correct file and directory permissions. 1.5 Appendix A.5: Conclusion The penetration test was successful in identifying and exploiting vulnerabilities in the "PumpkinFestival" virtual machine, with root access achieved and all PumpkinTokens collected. The test revealed several vulnerabilities in the system, including weak passwords, hidden tokens, and misconfigured permissions. The findings and recommendations provided in this report should be used to enhance the system’s security. Rights and permissions Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. Reprints and permissions About this article Check for updates. Verify currency and authenticity via CrossMark Cite this article Hilario, E., Azam, S., Sundaram, J. et al. Generative AI for pentesting: the good, the bad, the ugly. Int. J. Inf. Secur. 23, 2075–2097 (2024). https://doi.org/10.1007/s10207-024-00835-x Download citation Published: 15 March 2024 Version of record: 15 March 2024 Issue date: June 2024 DOI: https://doi.org/10.1007/s10207-024-00835-x Share this article Anyone you share the following link with will be able to read this content: Sorry, a shareable link is not currently available for this article. Provided by the Springer Nature SharedIt content-sharing initiative Keywords Cyber security Generative AI Large language models Penetration testing ChatGPT 3.5 Profiles Khwaja Imran Mohammed View author profile

Cited by

Sorted by citations

Related

Authors
Hilario, E., Azam, S., Sundaram, J., Imran Mohammed, K., & Shanmugam, B.
Affiliations
  • Charles Darwin University (AU)
  • Christ Academy Institute for Advanced Studies (IN)
Corresponding author
Shanmugam, Bharanidharan
ORCID
Venue
International Journal of Information Security
Publisher
Springer Science and Business Media LLC
Published
2024-03-15
Volume
23
Issue
3
Pages
2075-2097
ISSN
  • 1615-5262 (ISSN-L)
  • 1615-5270
DOI
10.1007/s10207-024-00835-x
OpenAlex
W4392848643
Cited by
49
References
13
Citations by year
2026: 14 · 2025: 22 · 2024: 12 · 2023: 1
FWCI
18.10
Type
journal article
Language
en
Open access
Licence
Locations
International Journal of Information Security (publishedVersion, cc-by)
Funders
Charles Darwin University
Topics
  • Physical Sciences › Computer Science › Information Systems › Web Application Security Vulnerabilities
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
Keywords
generative AI, large language models, penetration testing, ChatGPT, AI tool evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review