The DOI matches its Crossref record; 405 of the 405 references it cites were found in a registry or an index; 63 of the 193 sources its claims cite were checked for support.
When LLMs meet cybersecurity: a systematic literature review
Open accessAbstract
The rapid development of large language models (LLMs) has opened new avenues across various fields, including cybersecurity, which faces an evolving threat landscape and demand for innovative technologies. Despite initial explorations into the application of LLMs in cybersecurity, there is a lack of a comprehensive overview of this research area. This paper addresses this gap by providing a systematic literature review, covering the analysis of over 300 works, encompassing 25 LLMs and more than 10 downstream scenarios. Our comprehensive overview addresses three key research questions: the construction of cybersecurity-oriented LLMs, the application of LLMs to various cybersecurity tasks, the challenges and further research in this area.
Overview
CoChat Summary
## TL;DR
This is a systematic literature review of LLMs in cybersecurity, covering more than 300 papers, 25 LLMs, and more than 10 downstream scenarios. It concludes that LLMs are promising for cybersecurity tasks, but their own vulnerabilities, such as jailbreaks and other attacks, remain a major concern.
## Key findings
- The review analyzes “over 300 works,” spanning “25 LLMs” and “more than 10 downstream scenarios.”
- It identifies three main research questions: how to construct cybersecurity-oriented domain LLMs, what cybersecurity applications LLMs support, and what challenges/future research remain.
- The paper reports that LLMs have “great potential” across tasks including threat intelligence, vulnerability detection, secure code generation, program repair, binary analysis, IT operations, anomaly detection, and LLM-assisted attacks.
- It highlights concrete capability benchmarks and datasets for model selection, including CyberBench (60,000+), CyberMetric (10,000), SecEval (2,000+), DebugBench (4,253), NYU CTF Dataset (200), Cybench (40), and CTIBench (2,500).
- It also reports task-specific results such as SecureFalcon achieving “up to 96%” detection accuracy, LLMAO using bidirectional adapters for fault localization, and SafeCoder improving code security by “approximately 30%.”
## Methods
- Systematic literature review of more than 300 academic papers since 2023, organized around three research questions.
- The review groups prior work into cybersecurity task categories and model-construction approaches, including continual pre-training (CPT), supervised fine-tuning (SFT), full-parameter fine-tuning, and parameter-efficient fine-tuning (PEFT).
- It synthesizes studies, benchmarks, and datasets across domains such as vulnerability detection, threat intelligence, malware detection, fuzzing, program repair, secure code generation, and LLM attacks/defenses.
## Limitations
- The paper states that the field is still in its initial stages and that many studies are case studies without comprehensive methodology, which raises concerns about scalability and reproducibility.
- It explicitly notes that the literature lacks connectivity and in-depth analysis between studies.
- Evident from the text: much of the paper is a broad synthesis of cited work rather than a single unified empirical study, so many conclusions are aggregated from heterogeneous papers rather than one controlled dataset or experiment.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
59 Valid · 6 Not valid · 101 Not checked · 49 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Code-based LLMs (e.g., CodeLlama(Roziere etal. 2023) and StarCoder(Li etal. 2023d; Lozhkov et al. 2024)) are particularly well suited to address this demand.”
What the sources say
- Partly supportedcites Roziere et al. 2023 sourceExperimental — accuracy not yet measured
The Introduction identifies Code Llama as an LLM for code generation and infilling, and Related work discusses code-understanding tasks. This supports its use as an example of a code-based LLM. The Red teaming section evaluates responses to malicious-code requests, but does not establish that Code Llama can generate secure code or is particularly well suited to that security-focused demand. (LLM, conf=0.82)
- Not matchedcites Li et al. 2023d
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites Lozhkov et al. 2024 sourceExperimental — accuracy not yet measured
The Introduction identifies StarCoder as a code LLM and says code LLMs can assist with writing, editing, quality assurance, and bug detection. That supports the example and much of the claimed capability, but not the claim that they are particularly suited to generating secure code: the Results for the security benchmark report high insecure-program rates for StarCoder2-7B and -15B. Page numbers were not available in our copy. (LLM, conf=0.87)
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“To apply general LLMs to cybersecurity, researchers mainly employ two approaches: continual pre-training (CPT) and supervised fine-tuning (SFT).”
What the sources say
- Partly supportedcites Çağatay Yıldız et al. 2024 sourceExperimental — accuracy not yet measured
The Introduction and §5 Related work divide continual learning for LLMs into continual domain-adaptive pretraining and continual fine-tuning. They do not establish that these are the two main approaches to applying general LLMs to cybersecurity, and continual fine-tuning is not necessarily supervised fine-tuning. (LLM, conf=0.86)
- Not matchedcites Zhang et al. 2024e
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Wu et al. 2024c
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“SECURE (Bhusal etal. 2024) is a benchmark designed to assess LLMs' performance in realistic cybersecurity scenarios, which includes 6 datasets to evaluate the capabilities of knowledge extraction, understanding, and reasoning in the Industrial Control System scenarios.”
What the sources say
- Supportedcites Bhusal et al. 2024 source
The benchmark overview describes SECURE as evaluating LLMs in real-world cybersecurity scenarios and lists six datasets covering extraction, understanding, and reasoning. The Limitations & Future Work section confirms that the six datasets are centered on ICS security. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“By comparing LLMs' performance with static analysis tools, the paper demonstrates that LLMs can effectively pinpoint malware with higher precision and lower false positive rates.”
What the sources say
- Supportedcites Zahan et al. (2024) source
The RQ1 Results report higher precision and fewer false positives for GPT-3 and GPT-4 than the CodeQL static-analysis baseline. The Summary of RQ1 likewise says both models had superior precision while static analysis had a high false-positive rate. No page number is available in our copy. (LLM, conf=0.93)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Their results suggest that this is indeed the case and that LLMs have great potential for application in vulnerability detection.”
What the sources say
- Supportedcites Shestov et al. 2024 source
The Conclusion (p. 14) reports that fine-tuned WizardCoder improved vulnerability-detection performance over a prior model, supporting the claim that LLMs have potential for this application. The Conclusion (p. 15) also notes that performance still falls short of what business use requires, so the potential is not yet fully realized. (LLM, conf=0.90)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
37 more claims · 9 with their cited sources checked
Create a free account to see all 215 claims and to verify the 18 references still open.
- It may state two separate claims as one. 100
- Probably not a substantial claim. 136
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Zhang et al. [99], in Springer's Cybersecurity, review over 300 works on LLMs across cybersecurity tasks and independently identify hallucination, prompt-injection exposure, and evaluation immaturity as the dominant open problems”
From Agentic AI and Large Language Models for Autonomous IoT Cybersecurity: A Systematic Survey, Taxonomy, and Research Roadmap (2026) · cites this paper as [99]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These Gen-1 AI agents can perform natural language processing efficiently: they summarize long cyber reports in accessible language to summarize unstructured security warnings, help the analyst make sense of them, and respond to analyst queries with contextual reasoning and the like [7], [24].”
From The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines (2026) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“but even with those beneficial capabilities, Gen-1 systems are still fundamentally restricted, and one of the reason is that they don't run external queries, can't fetch data, run code, or talk to security tools [7].”
From The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines (2026) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This scope accords with new studies suggesting that cyber-defense tasks increasingly involve multi-hop reasoning in communication with operational sources of data [7], [8].”
From The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines (2026) · cites this paper as [7]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“They remain suitable for summarizing, explaining, and threat-intel digestion [7]”
From The Evolution of Agentic AI in Cybersecurity: From Single LLM Reasoners to Multi-Agent Systems and Autonomous Pipelines (2026) · cites this paper as [7]
Experimental — accuracy not yet measured
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 1×
“Hallucination is a response generated by AI that contains false or misleading information presented as fact (Maynez et al. 2020; Ji et al. 2023).”
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Conformer: Convolution-augmented Transformer for Speech Recognition
Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., Wu, Y., et al. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-3015 · Cited by 3,150 · cited 2×
“It compares the original code with LLM-generated code to flag notable differences as anomalies for further inspection.”
Conformer: Convolution-augmented Transformer for Speech Recognition
Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Zhang, Z., Wu, Y., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.08100 · Cited by 3,150
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096 · cited 2×
“Existing evaluation of LLM cybersecurity capabilities can be divided into several categories: cybersecurity knowledge, secure code generation, IT operations, Cap-ture-the-Flag (CTF), and cyber intelligence, as listed in Table 3.”
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · arXiv · doi:10.48550/arxiv.2104.08691 · Cited by 3,092
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Tabassi, E. (2023) · doi:10.6028/nist.ai.100-1 · Cited by 2,951
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604
Practical and ethical challenges of large language models in education: A systematic scoping review
Yan, L., Sha, L., Zhao, L., Li, Y., Martinez‐Maldonado, R., Chen, G., Li, X., Jin, Y., & Gašević, D. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13370 · Cited by 1,435 · cited 2×
“Such research enables LLMs to be tailored to specific challenges in various fields, thereby promoting progress and development in areas such as healthcare, law, education, and software engineering (Hou et al. 2023; Lai et al. 2024; Zhou et al. 2024b; Yan et al. 2024b; Li et al. 2023f; Zhao et al.…”
Research works printed but not linked: 33
- Alam MT, Bhusal D, Nguyen L et al (2024) CTIBench: a benchmark for evaluating LLMs in cyber threat intelligence. In: The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track
- Asmita, Oliinyk Y, Scott M et al (2024) Fuzzing busybox: leveraging LLM and crash reuse for embedded bug unearthing. In: Balzarotti D, Xu W (eds) 33rd USENIX Security Symposium, USENIX Security 2024, Philadelphia, PA, USA, August 14-16, 2024. USENIX Association, https://www.usenix.org/conference/usenixsecurity24/presentation/asmita
- Barnum S (2012) Standardizing cyber threat intelligence information with the structured threat information expression (stix). Mitre Corp 11:1–22
- Bozkurt A, Sharma RC (2023) Generative ai and prompt engineering: The art of whispering to let the genie out of the algorithmic world. Asian Journal of Distance Education 18(2):i–vii
- Brown T, Mann B, Ryder N et al (2020) Language models are few-shot learners. Adv Neural Inf Process Syst 33:1877–1901
- Carlini N, Tramer F, Wallace E et al (2021) Extracting training data from large language models. In: USENIX Security, pp 2633–2650
- Carlini N, Jagielski M, Zhang C et al (2022) The privacy onion effect: memorization is relative. In: NeurIPS, http://papers.nips.cc/paper_files/paper/2022/hash/564b5f8289ba846ebc498417e834c253-Abstract-Conference.html
- Chiang WL, Li Z, Lin Z et al (2023) Vicuna: an open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna lmsys org. 2(3):6. Accessed 14 April 2023
- Dettmers T, Pagnoni A, Holtzman A et al (2024) Qlora: efficient finetuning of quantized llms. Adv Neural Inf Process Syst. 36
- Elhage N, Nanda N, Olsson C et al (2021) A mathematical framework for transformer circuits. Trans Circuits Thread 1(1):12
- Fang C, Miao N, Srivastav S et al (2024a) Large language models for code analysis: Do llms really do their job? In: Balzarotti D, Xu W (eds) 33rd USENIX Security Symposium, USENIX Security 2024, Philadelphia, PA, USA, August 14-16, 2024. USENIX Association, https://www.usenix.org/conference/usenixsecurity24/presentation/fang
- Guo H, Yang J, Liu J et al (2024a) OWL: A large language model for IT operations. In: The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, https://openreview.net/forum?id=SZOQ9RKYJu
- He J, Vero M, Krasnopolska G et al (2024) Instruction tuning for secure code generation. In: Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, https://openreview.net/forum?id=MgTzMaYHvG
- Hu EJ, Shen Y, Wallis P et al (2022) Lora: Low-rank adaptation of large language models. In: The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, https://openreview.net/forum?id=nZeVKeeFYf9
- Huang L, Yu W, Ma W et al (2023a) A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions. ACM Trans Inf Syst
- Huang Y, Sun L, Wang H et al (2024) Trustllm: Trustworthiness in large language models. In: Forty-first International Conference on Machine Learning, ICML 2024,Vienna, Austria, July 21-27, 2024
- Huben R, Cunningham H, Smith LR et al (2023) Sparse autoencoders find highly interpretable features in language models. In: The Twelfth International Conference on Learning Representations
- Li R, Allal LB, Zi Y et al (2023d) Starcoder: may the source be with you! Trans Mach Learn Res 2023. https://openreview.net/forum?id=KoFOg41haE
- Li N, Pan A, Gopal A et al (2024c) The WMDP benchmark: Measuring and reducing malicious use with unlearning. In: Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net
- Lin Z, Cui J, Liao X et al (2024) Malla: Demystifying real-world large language model integrated malicious services. In: Balzarotti D, Xu W (eds) 33rd USENIX Security Symposium, USENIX Security 2024, Philadelphia, PA, USA, August 14-16, 2024. USENIX Association, https://www.usenix.org/conference/usenixsecurity24/presentation/lin-zilong
- Liu Z, Shi J, Buford JF (2024f) Cyberbench: A multi-task benchmark for evaluating large language models in cybersecurity. In: AAAI 2024 Workshop on Artificial Intelligence for Cyber Security
- Luo Z, Xu C, Zhao P et al (2024) Wizardcoder: Empowering code large language models with evol-instruct. In: The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, https://openreview.net/forum?id=UnUwSIgK5W
- Mijwil M, Aljanabi M et al (2023) Towards artificial intelligence-based cybersecurity: the practices and chatgpt generated ways to combat cybercrime. Iraqi J Comput Sci Math 4(1):65–70
- Nijkamp E, Pang B, Hayashi H et al (2023b) Codegen: An open large language model for code with multi-turn program synthesis. In: The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, https://openreview.net/forum?id=iaYcJKpY2B_
- Qin Y, Liang S, Ye Y et al (2024) Toolllm: Facilitating large language models to master 16000+ real-world apis. In: The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, https://openreview.net/forum?id=dHng2O0Jjr
- Radford A, Narasimhan K (2018) Improving language understanding by generative pre-training. https://api.semanticscholar.org/CorpusID:49313245
- Sandoval G, Pearce H, Nys T et al (2023) Lost at c: A user study on the security implications of large language model code assistants. In: 32nd USENIX Security Symposium (USENIX Security 23). USENIX Association, Anaheim, CA, pp 2205–2222, https://www.usenix.org/conference/usenixsecurity23/presentation/sandoval
- Sekhari A, Acharya J, Kamath G et al (2021) Remember what you want to forget: Algorithms for machine unlearning. In: NeurIPS, pp 18075–18086
- Tseng P, Yeh Z, Dai X et al (2024b) Using llms to automate threat intelligence analysis workflows in security operation centers. Preprint at arXiv:2407.13093 · arXiv
- Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. Adv Neural Inf Process Syst. 30
- Wang J, Hu X, Hou W et al (2024c) On the robustness of chatgpt: an adversarial and out-of-distribution perspective. IEEE Data Eng Bull 47(1):48–62
- Zhu S, Zhang R, An B et al (2024) Autodan: interpretable gradient-based adversarial attacks on large language models. In: First Conference on Language Modeling
- Zoph B, Raffel C, Schuurmans D et al (2022) Emergent abilities of large language models. TMLR
Cited by
Kheddar, H. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103347 · Cited by 161
Large language models: an overview of foundational architectures, recent trends, and a new taxonomy
Mienye, I. D., Jere, N., Obaido, G., Ogunruku, O. O., Esenogho, E., & Modisane, C. (2025) · Discover Applied Sciences · doi:10.1007/s42452-025-07668-w · Cited by 46
“For instance, Shafee et al. [134] and Zhang et al. [135] showed that LLMs can reduce the time needed for threat detection and response by automating common cybersecurity tasks, such as log analysis, generating alerts, and isolating compromised systems.”
Balasubramanian, P., Liyana, S., Sankaran, H., Sivaramakrishnan, S., Pusuluri, S., Pirttikangas, S., & Peltonen, E. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11338-z · Cited by 76
A Review of Large Language Models for Energy Systems: Applications, Challenges, and Future Prospects
Mirshekali, H., Shadi, M. R., Ladani, F. G., & Shaker, H. R. (2025) · IEEE Access · doi:10.1109/access.2025.3610994 · Cited by 96
“Security studies show these are common attack types and recommend safeguards like isolation, content provenance checks, and human verification before any policy or action is carried out [189], [190].”
Uddin, M., Irshad, M. S., Kandhro, I. A., Alanazi, F., Ahmed, F., Maaz, M., Hussain, S., & Ullah, S. S. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11219-5 · Cited by 81
Metadata
- Authors
- Zhang, J., Bu, H., Wen, H., Liu, Y., Fei, H., Xi, R., Li, L., Yang, Y., Zhu, H., & Meng, D.
- Affiliations
- Chinese Academy of Sciences (CN)
- Corresponding author
- Wen, Hui
- Zhu, Hongsong
- Venue
- Cybersecurity
- Published
- 2025-02-05
- Volume
- 8
- Issue
- 1
- ISSN
- 2523-3246 (ISSN-L)
- OpenAlex
- W4407163436
- Cited by
- 383
- References
- 156
- Citations by year
- 2026: 213 · 2025: 144
- FWCI
- 20.46
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Cybersecurity (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- Chinese Academy of Sciences
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- large language models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: When LLMs meet cybersecurity: a systematic literature review. https://app.cochat.ai/papers/10.1186/s42400-025-00361-w
- The paper
- Zhang, J., Bu, H., Wen, H., Liu, Y., Fei, H., Xi, R., Li, L., Yang, Y., Zhu, H., & Meng, D. (2025). When LLMs meet cybersecurity: a systematic literature review. Cybersecurity. https://doi.org/10.1186/s42400-025-00361-w
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.