The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacy
Open accessAbstract
Undoubtedly, the evolution of Generative AI (GenAI) models has been the highlight of digital transformation in the year 2022. As the different GenAI models like ChatGPT and Google Bard continue to foster their complexity and capability, it’s critical to understand its consequences from a cybersecurity perspective. Several instances recently have demonstrated the use of GenAI tools in both the defensive and offensive side of cybersecurity, and focusing on the social, ethical and privacy implications this technology possesses. This research paper highlights the limitations, challenges, potential risks, and opportunities of GenAI in the domain of cybersecurity and privacy. The work presents the vulnerabilities of ChatGPT, which can be exploited by malicious users to exfiltrate malicious information bypassing the ethical constraints on the model. This paper demonstrates successful example attacks like Jailbreaks, reverse psychology, and prompt injection attacks on the ChatGPT. The paper also investigates how cyber offenders can use the GenAI tools in developing cyber attacks, and explore the scenarios where ChatGPT can be used by adversaries to create social engineering attacks, phishing attacks, automated hacking, attack payload generation, malware creation, and polymorphic malware. This paper then examines defense techniques and uses GenAI tools to improve security measures, including cyber defense automation, reporting, threat intelligence, secure code generation and detection, attack identification, developing ethical guidelines, incidence response plans, and malware detection. We will also discuss the social, legal, and ethical implications of ChatGPT. In conclusion, the paper highlights open challenges and future directions to make this GenAI secure, safe, trustworthy, and ethical as the community understands its cybersecurity impacts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Generative adversarial networks
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020) · Communications of the ACM · doi:10.1145/3422622 · Cited by 18,384
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Towards a Human-like Open-Domain Chatbot
Adiwardana, D., Luong, M.-T., So, D. R., Hall, J., Fiedel, N., Thoppilan, R., Yang, Z., Kulshreshtha, A., Nemade, G., Lu, Y., et al. (2020) · arXiv · doi:10.48550/arxiv.2001.09977 · Cited by 614 · cited 1×
“Bard AI, initially introduced as Meena [63], has also undergone various refinements, demonstrating significant improvements in human-like conversational abilities.”
BackgroundWeaponized AI for cyber attacks
Yamin, M. M., Ullah, M., Ullah, H., & Katt, B. (2021) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2020.102722 · Cited by 154
Cited by
A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., & Zhang, Y. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100211 · Cited by 1,083
Security and Privacy Challenges of Large Language Models: A Survey
Das, B. C., Amini, M. H., & Wu, Y. (2025) · ACM Computing Surveys · doi:10.1145/3712001 · Cited by 445
Bengesi, S., El-Sayed, H., Sarker, M. K., Houkpati, Y., Irungu, J., & Oladunni, T. (2024) · IEEE Access · doi:10.1109/access.2024.3397775 · Cited by 362
Almatrafi, O., Johri, A., & Lee, H. (2024) · Computers and Education Open · doi:10.1016/j.caeo.2024.100173 · Cited by 341
Ali, O., Murray, P. A., Momin, M., Dwivedi, Y. K., & Malik, T. (2024) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2023.123076 · Cited by 317
Privacy and Security Concerns in Generative AI: A Comprehensive Survey
Golda, A., Mekonen, K., Pandey, A., Singh, A., Hassija, V., Chamola, V., & Sikdar, B. (2024) · IEEE Access · doi:10.1109/access.2024.3381611 · Cited by 264
Ferrara, E. (2024) · Journal of Computational Social Science · doi:10.1007/s42001-024-00250-1 · Cited by 259
When LLMs meet cybersecurity: a systematic literature review
Zhang, J., Bu, H., Wen, H., Liu, Y., Fei, H., Xi, R., Li, L., Yang, Y., Zhu, H., & Meng, D. (2025) · Cybersecurity · doi:10.1186/s42400-025-00361-w · Cited by 383
“Their experiments show that LLMs outperform traditional methods in generating test cases when a detailed description of the vulnerability, possible exploits, and code context are given.”
AI agents under threat: A survey of key security challenges and future pathways
Deng, Z., Guo, Y., Han, C., Ma, W., Xiong, J., Wen, S., & Xiang, Y. (2025) · ACM Computing Surveys · doi:10.1145/3716628 · Cited by 210
“To better understand the action threats, we present the action structure as follows:”
Digital deception: generative artificial intelligence in social engineering and phishing
Schmitt, M., & Flechais, I. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10973-2 · Cited by 198
Kurtz, G., Amzalag, M., Shaked, N., Zaguri, Y., Kohen-Vacs, D., Gal, E., Zailer, G., & Barak-Medina, E. (2024) · Education Sciences · doi:10.3390/educsci14050503 · Cited by 181
“One concern is the possibility of using these applications without proper attribution, which may undermine essential academic skills such as critical thinking, problem-solving, creativity, research abilities, and interpersonal communication [50,51].”
BackgroundMapping the ethics of generative AI: a comprehensive scoping review
Hagendorff, T. (2024) · Minds and Machines · doi:10.1007/s11023-024-09694-w · Cited by 141
Sai, S., Yashvardhan, U., Chamola, V., & Sikdar, B. (2024) · IEEE Access · doi:10.1109/access.2024.3385107 · Cited by 125
Zhang, R., Li, H., Meng, H., Zhan, J., Gan, H., & Lee, Y.-C. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713429 · Cited by 123
“Moreover, the scarcity of large-scale interactional data, due to their private and sensitive nature, limits deeper analysis.”
Akiba, D., & Fraboni, M. C. (2023) · Education Sciences · doi:10.3390/educsci13090885 · Cited by 115
Large Language Model‐Based Chatbots in Higher Education
Yigci, D., Eryilmaz, M., Yetisen, A. K., Tasoglu, S., & Ozcan, A. (2025) · Advanced Intelligent Systems · doi:10.1002/aisy.202400429 · Cited by 113
Zhang, X., Zhang, P., Shen, Y., Liu, M., Wang, Q., Gašević, D., & Fan, Y. (2024) · Frontiers of Digital Education · doi:10.1007/s44366-024-0028-5 · Cited by 83
StarCoder 2 and The Stack v2: The Next Generation
Lozhkov, A., Li, R., Allal, L. B., Cassano, F., Lamy-Poirier, J., Tazi, N., Tang, A., Pykhtar, D., Liu, J., Wei, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2402.19173 · Cited by 74
Yigit, Y., Ferrag, M. A., Ghanem, M. C., Sarker, I. H., Maglaras, L. A., Chrysoulas, C., Moradpoor, N., Tihanyi, N., & Janicke, H. (2025) · Sensors · doi:10.3390/s25061666 · Cited by 69
“Research in [125] demonstrates that ChatGPT could be used to generate social engineering attacks, phishing schemes, automated hacking, attack payloads, malware, and polymorphic malware.”
University Students’ Insights of Generative Artificial Intelligence (AI) Writing Tools
Gasaymeh, A.-M. M., Beirat, M. A., & Abu Qbeita, A. A. (2024) · Education Sciences · doi:10.3390/educsci14101062 · Cited by 74
(2024) · doi:10.1109/sp54263.2024.00182 · Cited by 62
“Investigations by Gupta et al. [50] and Derner et al. [51] have unveiled vulnerabilities in ChatGPT that can be harnessed to generate malware.”
BackgroundGenerative AI for pentesting: The good, the bad, the ugly
Hilario, E., Azam, S., Sundaram, J., Imran Mohammed, K., & Shanmugam, B. (2024) · International Journal of Information Security · doi:10.1007/s10207-024-00835-x · Cited by 49
Meng, X., & Liu, J. (2025) · Online Information Review · doi:10.1108/oir-06-2024-0375 · Cited by 46
Patel, K., Shah, M., Qureshi, K. M., & Qureshi, M. R. N. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11435-z · Cited by 42
Personality and acceptance as predictors of ChatGPT use
de Winter, J., Dodou, D., & Eisma, Y. B. (2024) · Discover Psychology · doi:10.1007/s44202-024-00161-2 · Cited by 43
AI-Augmented SOC: A Survey of LLMs and Agents for Security Automation
Srinivas, S., Kirk, B., Zendejas, J., Espino, M., Boskovich, M., Bari, A., Dajani, K., & Alzahrani, N. (2025) · Journal of Cybersecurity and Privacy · doi:10.3390/jcp5040095 · Cited by 39
Ferrara, E. (2023) · doi:10.2139/ssrn.4614223 · Cited by 266
Banerjee, P., Srivastava, A. K., Adjeroh, D. A., Reddy, R., & Karimian, N. (2025) · IEEE Access · doi:10.1109/access.2024.3524102 · Cited by 32
Lim, J. S., Shin, D., Lee, C., Kim, J., & Zhang, J. (2025) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838151.2025.2487679 · Cited by 32
Jelodar, H., Bai, S., Hamedi, P., Mohammadian, H., Razavi-Far, R., & Ghorbani, A. (2026) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2026.104390 · Cited by 27
Benchmark Data Contamination of Large Language Models: A Survey
Xu, C., Guan, S., Greene, D., & Kechadi, M.-T. (2024) · arXiv · doi:10.48550/arxiv.2406.04244 · Cited by 22
Enhancing DevSecOps practice with Large Language Models and Security Chaos Engineering
Bedoya, M., Palacios, S., Díaz-López, D., Laverde, E., & Nespoli, P. (2024) · International Journal of Information Security · doi:10.1007/s10207-024-00909-w · Cited by 19
Generative AI and data protection
Ruschemeier, H. (2025) · Cambridge Forum on AI: Law and Governance · doi:10.1017/cfl.2024.2 · Cited by 16
Damacena Duarte, J., Cândido, G. D., De Britto Filho, J. R. A., Souza Neto, J., Da Costa, E. J., Da Costa, J. P. J., & Peotta De Melo, L. (2026) · IEEE Access · doi:10.1109/access.2026.3656849 · Cited by 15
Malla: Demystifying Real-world Large Language Model Integrated Malicious Services
Lin, Z., Cui, J., Liao, X., & Wang, X. (2024) · arXiv · doi:10.48550/arxiv.2401.03315 · Cited by 13
“Past research showcased how LLMs can be weaponized across diverse domains, such as misinformation propagation [91,102,125], deepfake user profile creation [109], spear phishing campaigns [97, 98], attack and malware generation [97] and the generation of hateful memes [119].”
BackgroundMethodReti, D., Becker, N., Angeli, T., Chattopadhyay, A., Schneider, D., Vollmer, S., & Schotten, H. D. (2024) · Proceedings of the 11th ACM Workshop on Adaptive and Autonomous Cyber Defense · doi:10.1145/3689935.3690394 · Cited by 13
“…often indistinguishable from one written by a human, was quickly abused for misinformation attacks as it could be used to automatically generate targeted phishing emails or publish false information through social media on a large scale, demonstrating the power of LLMs to deceive humans [14, 40].”
BackgroundA Lifecycle-Oriented Survey of Emerging Threats and Vulnerabilities in Large Language Models
de Maio, C., Di Gisi, M., Fenza, G., Gallo, M., & Loia, V. (2025) · IEEE Access · doi:10.1109/access.2025.3619764 · Cited by 10
Honeypot's Best Friend? Investigating ChatGPT's Ability to Evaluate Honeypot Logs
Ozkok, M. B., Birinci, B., Cetin, O., Arief, B., & Hernandez-Castro, J. (2024) · European Interdisciplinary Cybersecurity Conference · doi:10.1145/3655693.3655716 · Cited by 8
“Gupta et al. [3] explains how attackers can use ChatGPT to exploit the vulnerability, along with defense techniques such as cyber defense automation, threat intelligence, attack explanation, and malware detection.”
BackgroundUsman, Y., Upadhyay, A., Gyawali, P., & Chataut, R. (2024) · arXiv · doi:10.48550/arxiv.2408.12806 · Cited by 2
“As a result, traditional cybersecurity solutions are becoming inadequate in detecting and mitigating these advanced cyberattacks [1].”
BackgroundThe Role of AI Chatbots in Facilitating Online Harm: A Systematic Review
Amidu, G. (2025) · doi:10.21203/rs.3.rs-8427928/v1 · Cited by 1
Metadata
- Authors
- Gupta, M., Akiri, C., Aryal, K., Parker, E., & Praharaj, L.
- Venue
- IEEE Access
- Published
- 2023
- Volume
- 11
- Pages
- 80218-80245
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4385452929
- Cited by
- 786
- References
- 5
- FWCI
- 69.05
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.