The DOI matches its Crossref record; 41 of the 49 references it cites were found in a registry or an index; 9 of the 13 sources its claims cite were checked for support.
HuntGPT: Integrating Machine Learning-Based Anomaly Detection and Explainable AI with Large Language Models (LLMs)
Open accessAbstract
Machine learning (ML) methods for network anomaly detection are emerging as effective proactive strategies in threat hunting, substantially reducing the time required for threat detection and response. However, the challenges in training and maintaining ML models, coupled with frequent false positives, diminish their acceptance and trustworthiness. In response, Explainable AI (XAI) techniques have been introduced to enable cybersecurity operations teams to assess alerts generated by AI systems more confidently. Despite these advancements, XAI tools have encountered limited acceptance from incident responders and have struggled to meet the decision-making needs of both analysts and model maintainers. Large Language Models (LLMs) offer a unique approach to tackling these challenges. Through tuning, LLMs have the ability to discern patterns across vast amounts of information and meet varying functional requirements. In this research, we introduce the development of HuntGPT, a specialized intrusion detection dashboard created to implement a Random Forest classifier trained utilizing the KDD99 dataset. The tool incorporates XAI frameworks like SHAP and Lime, enhancing user-friendliness and intuitiveness of the model. When combined with a GPT-3.5 Turbo conversational agent, HuntGPT aims to deliver detected threats in an easily explainable format, emphasizing user understanding and offering a smooth interactive experience. We investigate the system’s comprehensive architecture and its diverse components, assess the prototype’s technical accuracy using the Certified Information Security Manager (CISM) Practice Exams, and analyze the quality of response readability across six unique metrics. Our results indicate that conversational agents, underpinned by LLM technology and integrated with XAI, can enable a robust mechanism for generating explainable and actionable AI solutions, especially within the realm of intrusion detection systems.
Overview
CoChat Summary
TL;DR
HuntGPT is an intrusion-detection dashboard that combines a Random Forest classifier trained on KDD99, SHAP/LIME explanations, and a GPT-3.5 Turbo conversational agent to make network anomaly alerts more explainable and actionable. Evaluation shows GPT-3.5 Turbo achieved 72–82.5% success on cybersecurity certification practice exams, while generated explanations and chatbot answers were mostly at graduate-level readability.
Key findings
- GPT-3.5 Turbo’s cybersecurity knowledge was tested on three practice exams: CISM Certified Information Security Manager Practice Exams (40 questions, 82.5% success), ISACA official CISM practice quiz (10 questions, 80%), and ISACA official cybersecurity fundamentals practice quiz (25 questions, 72%).
- Readability analysis of 20 generated anomaly explanations and 20 chat logs found mostly graduate-level text: Flesch-Kincaid Grade Level 15.7 for explanations and 14.9 for chatbot answers; Flesch Reading Ease 22.7 and 23.9; Dale Chall 12.6 and 12.11; Automated Readability Index 16.3 and 15.9; Coleman Liau Index 15.3 and 16.3; Linsear Write 17.4 and 15.8.
- The paper reports that the chatbot produced actionable responses, with example queries about preventing DoS attacks, firewalls, free firewall options, installing ZoneAlarm, and configuring a small home network.
- The prototype is presented as integrating ML anomaly detection, XAI, and LLM-based conversation to improve analyst understanding and trust in intrusion detection.
Methods
- Design: HuntGPT is organized into three layers—an analytics engine for packet analysis, Elasticsearch/S3 for data storage, and a Gradio dashboard UI integrated with OpenAI’s Language Model API.
- Detection and explanation: a Random Forest classifier trained on the KDD99 dataset classifies network flows; SHAP and LIME generate explanations, plots, and JSON documents stored in Elasticsearch.
- Evaluation: technical accuracy was assessed using CISM and ISACA practice exams; response quality was assessed on 20 generated explanations and 20 chat logs, each with approximately six questions, using six readability formulas from the Python py-readability-metrics library.
Limitations
- The paper states future work is needed to refine the ML model and incorporate real-time detection capabilities, and to embed actionable AI so the chatbot can issue active commands directly to a SIEM component.
- Evident from the text: evaluation relies on certification practice exams and readability metrics rather than operational SOC deployment, so real-world false-positive reduction, analyst decision accuracy, and production performance are not measured.
- Evident from the text: the response-quality evaluation uses a small sample of 20 generated explanations and 20 chat logs, and the KDD99 dataset is an older benchmark, which may limit generalization to current network traffic.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
7 Valid · 2 Not valid · 3 Not checked · 18 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Evidently, the infusion of machine learning into CTH tools has notably increased the incidence of false positives in real-world operational environments [9].”
What the sources say
- Partly supportedcites [9] Cavallaro, 2023, Are Machine Learning Models for Malware Detection Ready for Prime Time?, IEEE Secur. Priv. sourceExperimental — accuracy not yet measured
The “Cannot Reproduce?” section (p. 1) says that machine-learning malware classifiers can misclassify benign software as malicious and perform poorly in practical settings. It does not show that introducing machine learning increased false-positive incidence, nor does it establish that claim for cyber threat-hunting tools generally. (LLM, conf=0.87)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Similarly, a recent systematic review [31] highlighted that research on XAI for cybersecurity shows that many XAI applications are crafted without a thorough understanding of their integration into analyst workflows.”
What the sources say
- Partly supportedcites [31] Han, D., Wang, Z., Chen, W., Zhong, Y., Wang, S., Zhang, H., Yang, J., Shi, X., and Yin, X. (2021, January 15–19). Deepaid: Interpreting and improving deep learning-based anomaly detection in security applications. Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, Virtual. sourceExperimental — accuracy not yet measured
Section 3.1 says security systems need expert feedback and that black-box models make it difficult for experts to interact with them. But DeepAID is not a systematic review, and the cited passages do not establish that many XAI applications were designed without understanding analyst workflows. (LLM, conf=0.83)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Based on the results presented in Table III, GPT-3.5turbo demonstrates substantial proficiency in cybersecurity knowledge, with success rates ranging from 72% to 82.5% across varied and reputable exams.”
- Also stated in: § V. CONCLUSION AND FUTURE WORK · p. 8 · plain
“The model demonstrated substantial proficiency in cybersecurity knowledge, as evident from its success rates ranging between 72% and 82.5% on various reputable standardized certification exams in cybersecurity.”
What the sources say
- Not matchedcites [14] Gregory, P. (2023). CISM Certified Information Security Manager Practice Exams, McGraw Hill LLC. [2nd ed.].
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Could not verifycites [43] Chu, 2023, Assessing the Benefits of ChatGPT for Business: An Empirical Study on Organizational Performance, IEEE Access sourceExperimental — accuracy not yet measured
The Instrument Development section describes a survey of office workers, and the Discussion addresses ChatGPT’s effects on organizational performance. Searches for GPT-3.5-turbo, cybersecurity certification exams, and the 72%–82.5% success-rate range found no passage in our copy addressing the claimed exam results. (LLM, conf=0.78)
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § V. CONCLUSION AND FUTURE WORK · p. 8 · plain
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“However, there is room for improvement, particularly in the ISACA official cybersecurity fundamentals practice quiz where the model achieved a lower success rate of 72%.”
- Also stated in: § V. CONCLUSION AND FUTURE WORK · p. 8 · plain
“However, these results also highlight areas for enhancement, mainly focusing on improving proficiency in fundamental cybersecurity concepts where the model achieved a lower success rate.”
What the sources say
- Could not verifycites [43] Chu, 2023, Assessing the Benefits of ChatGPT for Business: An Empirical Study on Organizational Performance, IEEE Access sourceExperimental — accuracy not yet measured
The Instrument Development and Data Collection sections describe a survey of ChatGPT users, not a cybersecurity quiz. Searches for the ISACA practice quiz, GPT-3.5-turbo, and a 72% success rate found no passage addressing the claimed exam result in our copy. (LLM, conf=0.76)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § V. CONCLUSION AND FUTURE WORK · p. 8 · plain
In the paper
“In this research, we introduce the development of HuntGPT, a specialized intrusion detection dashboard created to implement a Random Forest classifier trained utilizing the KDD99 dataset.”
- Also stated in: § I. INTRODUCTION · p. 2 · plain
“The prototype is designed to perform analysis on network traffic, utilizing a Random Forest classifier as the anomaly detection model. This model, trained utilizing the KDD99 dataset [13], is deployed to classify the acquired packets systematically.”
What the sources say
- Not matchedcites [13] Stolfo, S., Fan, W., Lee, W., Prodromidis, A., and Chan, P. (1999). KDD Cup 1999 Data, UCI Machine Learning Repository.
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § I. INTRODUCTION · p. 2 · plain
6 more claims
Create a free account to see all 30 claims and every source verdict.
- Probably not a substantial claim. 16
- It may state two separate claims as one. 13
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ B. Response Quality Analysis · p. 6
“ISACA official CISM practice Quiz. A free practice quiz including questions from ISACA's test prep solutions that are the same level of difficulty of ISACA's official CISM exam [43].”
§ REFERENCES · p. 10
“- [43] Wikipedia contributors. ISACA - Wikipedia, The Free Encyclopedia .”
Citations
Cites
Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023) · JMIR Medical Education · doi:10.2196/45312 · Cited by 2,078
XAI—Explainable artificial intelligence
Gunning, D., Stefik, M., Choi, J., Miller, T., Stumpf, S., & Yang, G.-Z. (2019) · Science Robotics · doi:10.1126/scirobotics.aay7120 · Cited by 2,053
Network Anomaly Detection: Methods, Systems and Tools
Bhuyan, M. H., Bhattacharyya, D. K., & Kalita, J. K. (2014) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2013.052213.00046 · Cited by 1,348
Federated-Learning-Based Anomaly Detection for IoT Security Attacks
Mothukuri, V., Khare, P., Parizi, R. M., Pouriyeh, S., Dehghantanha, A., & Srivastava, G. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3077803 · Cited by 722
DÏoT: A Federated Self-learning Anomaly Detection System for IoT
Nguyen, T. D., Marchal, S., Miettinen, M., Fereidooni, H., Asokan, N., & Sadeghi, A.-R. (2019) · 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00080 · Cited by 663
Anomaly detection in IP networks
Thottan, M., & Ji, C. (2003) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2003.814797 · Cited by 541
The Roadmap to 6G Security and Privacy
Porambage, P., Gur, G., Osorio, D. P. M., Liyanage, M., Gurtov, A., & Ylianttila, M. (2021) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2021.3078081 · Cited by 416
Chained Anomaly Detection Models for Federated Learning: An Intrusion Detection Case Study
Preuveneers, D., Rimmer, V., Tsingenopoulos, I., Spooren, J., Joosen, W., & Ilie-Zudor, E. (2018) · Applied Sciences · doi:10.3390/app8122663 · Cited by 323
Prompting GPT-3 To Be ReliablePreprint
Si, C., Gan, Z., Yang, Z., Wang, S., Wang, J., Boyd-Graber, J. L., & Wang, L. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.09150 · Cited by 292
Explainable artificial intelligence for cybersecurity: a literature survey
Charmet, F., Tanuwidjaja, H. C., Ayoubi, S., Gimenez, P.-F., Han, Y., Jmila, H., Blanc, G., Takahashi, T., & Zhang, Z. (2022) · Annals of Telecommunications · doi:10.1007/s12243-022-00926-7 · Cited by 167
Cited by
A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., & Zhang, Y. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100211 · Cited by 1,083
The application of large language models in medicine: A scoping review
Meng, X., Yan, X., Zhang, K., Liu, D., Cui, X., Yang, Y., Zhang, M., Cao, C., Wang, J., Wang, X., et al. (2024) · iScience · doi:10.1016/j.isci.2024.109713 · Cited by 414
When LLMs meet cybersecurity: a systematic literature review
Zhang, J., Bu, H., Wen, H., Liu, Y., Fei, H., Xi, R., Li, L., Yang, Y., Zhu, H., & Meng, D. (2025) · Cybersecurity · doi:10.1186/s42400-025-00361-w · Cited by 383
Friha, O., Amine Ferrag, M., Kantarci, B., Cakmak, B., Ozgun, A., & Ghoualmi-Zine, N. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3456549 · Cited by 189
Kheddar, H. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103347 · Cited by 161
Metadata
- Authors
- Ali, T., Kostakos, P., & Sheikhi, S.
- Affiliations
- University of Oulu (FI)
- Venue
- Telecom
- Publisher
- MDPI AG
- Published
- 2026-06-08
- Volume
- 7
- Issue
- 3
- Pages
- 73
- ISSN
- 2673-4001 (ISSN-L)
- OpenAlex
- W4387210439
- Cited by
- 104
- References
- 48
- Citations by year
- 2026: 33
- FWCI
- 91.18
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Telecom (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- European Commission (101021911, 318927)
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- network anomaly detection, threat hunting, explainable AI, large language models, intrusion detection systems, random forest, SHAP, lime, GPT-3.5-turbo, chatbots, KDD Cup 1999 dataset, false positives, incident response
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: HuntGPT: Integrating Machine Learning-Based Anomaly Detection and Explainable AI with Large Language Models (LLMs). https://app.cochat.ai/papers/10.3390/telecom7030073
- The paper
- Ali, T., Kostakos, P., & Sheikhi, S. (2026). HuntGPT: Integrating Machine Learning-Based Anomaly Detection and Explainable AI with Large Language Models (LLMs). Telecom. https://doi.org/10.3390/telecom7030073
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.