CoChat
Sign Up
PreprintCited by 9Topic ModelingAdversarial Robustness in Machine LearningInformation and Cyber Security

The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.

SecQA: A Concise Question-Answering Dataset for Evaluating Large Language Models in Computer Security

Open access
Liu, Zefang·2023·arXiv
DOI 10.48550/arxiv.2312.15838arXiv 2312.15838
Open this paper in CoChat — free accountOpen PDF

Abstract

In this paper, we introduce SecQA, a novel dataset tailored for evaluating the performance of Large Language Models (LLMs) in the domain of computer security. Utilizing multiple-choice questions generated by GPT-4 based on the "Computer Systems Security: Planning for Success" textbook, SecQA aims to assess LLMs' understanding and application of security principles. We detail the structure and intent of SecQA, which includes two versions of increasing complexity, to provide a concise evaluation across various difficulty levels. Additionally, we present an extensive evaluation of prominent LLMs, including GPT-3.5-Turbo, GPT-4, Llama-2, Vicuna, Mistral, and Zephyr models, using both 0-shot and 5-shot learning settings. Our results, encapsulated in the SecQA v1 and v2 datasets, highlight the varying capabilities and limitations of these models in the computer security context. This study not only offers insights into the current state of LLMs in understanding security-related content but also establishes SecQA as a benchmark for future advancements in this critical research area.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2312.15838

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

  • Mistral 7B

    Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. D. L., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.06825 · Cited by 340

Show all 1 references

Research works printed but not linked: 18

  1. Alam, M. T.; Bhusal, D.; Park, Y.; and Rastogi, N. 2022. CyNER: A Python Library for Cybersecurity Named Entity Recognition. arXiv preprint arXiv:2204.05754. · arXiv
  2. Andrew, Y.; Lim, C.; and Budiarto, E. 2022. Mapping Linux Shell Commands to MITRE ATT&CK using NLP-Based Approach. In 2022 International Conference on Electrical Engineering and Informatics (ICELTICs), 37-42. IEEE.
  3. bench authors, B. 2023. Beyond the Imitation Game: Quan- tifying and extrapolating the capabilities of language mod- els. Transactions on Machine Learning Research.
  4. Das, S. S.; Serra, E.; Halappanavar, M.; Pothen, A.; and Al- Shaer, E. 2021. V2W-BERT: A Framework for Effective Hierarchical Multiclass Classification of Software Vulnera- bilities. In 2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA), 1-12. IEEE.
  5. Ferrag, M. A.; Ndhlovu, M.; Tihanyi, N.; Cordeiro, L. C.; Debbah, M.; and Lestable, T. 2023. Revolutionizing Cy- ber Threat Detection with Large Language Models. arXiv preprint arXiv:2306.14263. · arXiv
  6. Hendrycks, D.; Burns, C.; Basart, S.; Zou, A.; Mazeika, M.; Song, D.; and Steinhardt, J. 2020. Measuring Mas- sive Multitask Language Understanding. arXiv preprint arXiv:2009.03300. · arXiv
  7. Kaddour, J.; Harris, J.; Mozes, M.; Bradley, H.; Raileanu, R.; and McHardy, R. 2023. Challenges and Applications of Large Language Models. arXiv preprint arXiv:2307.10169. · arXiv
  8. Liang, P.; Bommasani, R.; Lee, T.; Tsipras, D.; Soylu, D.; Yasunaga, M.; Zhang, Y.; Narayanan, D.; Wu, Y.; Kumar, A.; et al. 2022. Holistic Evaluation of Language Models. arXiv preprint arXiv:2211.09110. · arXiv
  9. Liu, Z.; and Buford, J. 2023. Anomaly Detection of Command Shell Sessions based on DistilBERT: Unsu- pervised and Supervised Approaches. arXiv preprint arXiv:2310.13247. · arXiv
  10. OpenAI. 2023. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774. · arXiv
  11. Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; et al. 2022. Training Language Models to Follow Instruc- tions with Human Feedback. Advances in Neural Informa- tion Processing Systems, 35: 27730-27744.
  12. Rahali, A.; and Akhloufi, M. A. 2021. MalBERT: Using Transformers for Cybersecurity and Malicious Software De- tection. arXiv preprint arXiv:2103.03806. · arXiv
  13. Rahali, A.; and Akhloufi, M. A. 2023. MalBERTv2: Code Aware BERT-Based Model for Malware Identification. Big Data and Cognitive Computing, 7(2): 60.
  14. Shahid, M. R.; and Debar, H. 2021. CVSS-BERT: Explain- able Natural Language Processing to Determine the Severity of a Computer Security Vulnerability from its Description. In 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), 1600-1607. IEEE.
  15. Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; et al. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv preprint arXiv:2307.09288. · arXiv
  16. Tunstall, L.; Beeching, E.; Lambert, N.; Rajani, N.; Rasul, K.; Belkada, Y.; Huang, S.; von Werra, L.; Fourrier, C.; Habib, N.; et al. 2023. Zephyr: Direct Distillation of LM Alignment. arXiv preprint arXiv:2310.16944. · arXiv
  17. Wang, X.; He, S.; Xiong, Z.; Wei, X.; Jiang, Z.; Chen, S.; and Jiang, J. 2022. APTNER: A Specific Dataset for NER Missions in Cyber Threat Intelligence Field. In 2022 IEEE 25th International Conference on Computer Supported Co- operative Work in Design (CSCWD), 1233-1238. IEEE.
  18. Zheng, L.; Chiang, W.-L.; Sheng, Y.; Zhuang, S.; Wu, Z.; Zhuang, Y.; Lin, Z.; Li, Z.; Li, D.; Xing, E. P.; Zhang, H.; Gonzalez, J. E.; and Stoica, I. 2023. Judg- ing LLM-as-a-judge with MT-Bench and Chatbot Arena. arXiv:2306.05685. · arXiv

Cited by

Sorted by citations

Metadata

Authors
Liu, Z.
Affiliations
Georgia Institute of Technology (US)
Corresponding author
Liu, Zefang
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2312.15838
arXiv
2312.15838
OpenAlex
W4390306856
Cited by
9
References
1
Citations by year
2025: 3 · 2024: 6
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Information Systems › Information and Cyber Security
Keywords
question answering dataset, large language models, cybersecurity, LLM evaluation, benchmark, multiple-choice questions, GPT-4, security principles, LLM comparison

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review