CoChat
Sign Up
PreprintCited by 2Cryptographic Implementations and SecuritySecurity and Verification in ComputingWeb Application Security Vulnerabilities

ZeroLeak: Using LLMs for Scalable and Cost Effective Side-Channel Patching

Open access
Tol, M. Caner, Sunar, Berk·2023·arXiv
DOI 10.48550/arxiv.2308.13062arXiv 2308.13062
Open this paper in CoChat — free accountOpen PDF

Abstract

Security critical software, e.g., OpenSSL, comes with numerous side-channel leakages left unpatched due to a lack of resources or experts. The situation will only worsen as the pace of code development accelerates, with developers relying on Large Language Models (LLMs) to automatically generate code. In this work, we explore the use of LLMs in generating patches for vulnerable code with microarchitectural side-channel leakages. For this, we investigate the generative abilities of powerful LLMs by carefully crafting prompts following a zero-shot learning approach. All generated code is dynamically analyzed by leakage detection tools, which are capable of pinpointing information leakage at the instruction level leaked either from secret dependent accesses or branches or vulnerable Spectre gadgets, respectively. Carefully crafted prompts are used to generate candidate replacements for vulnerable code, which are then analyzed for correctness and for leakage resilience. From a cost/performance perspective, the GPT4-based configuration costs in API calls a mere few cents per vulnerability fixed. Our results show that LLM-based patching is far more cost-effective and thus provides a scalable solution. Finally, the framework we propose will improve in time, especially as vulnerability detection tools and LLMs mature.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2308.13062

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • ZeroLeak guides LLMs to generate patches for side-channel vulnerabilities through zero-shot learning, and these patches are inspected by dynamic analysis tools to ensure correct function and prevent information leakage.

    “ZeroLeak guides LLMs to generate patches for specific vulnerabilities through zero-shot learning. Once generated, these patches are inspected by dynamic analysis tools to ensure that they not only function correctly, but also prevent information leakage.”

    From When LLMs meet cybersecurity: a systematic literature review (2025) · cites this paper as Tol and Sunar (2023)

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 35

  1. AHMAD, B., THAKUR, S., TAN, B., KARRI, R., AND PEARCE, H. Fixing hardware security bugs with large language models. arXiv preprint arXiv:2302.01215 (2023). · arXiv
  2. ANTONOPOULOS, T., GAZZILLO, P., HICKS, M., KOSKINEN, E., TERAUCHI, T., AND WEI, S. Decomposition instead of self- composition for proving the absence of timing channels. ACM SIGPLAN Notices 52, 6 (2017), 362-375.
  3. BROWN, T., MANN, B., RYDER, N., SUBBIAH, M., KAPLAN, J. D., DHARIWAL, P., NEELAKANTAN, A., SHYAM, P., SASTRY, G., ASKELL, A., ET AL. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877- 1901.
  4. CANELLA, C., GENKIN, D., GINER, L., GRUSS, D., LIPP, M., MINKIN, M., MOGHIMI, D., PIESSENS, F., SCHWARZ, M., SUNAR, B., VAN BULCK, J., AND YAROM, Y. Fallout: Leaking data on meltdown-resistant cpus. In Proceedings of the ACM SIGSAC Con- ference on Computer and Communications Security (CCS) (2019), ACM.
  5. CANELLA, C., VAN BULCK, J., SCHWARZ, M., LIPP, M., VON BERG, B., ORTNER, P., PIESSENS, F., EVTYUSHKIN, D., AND GRUSS, D. A systematic evaluation of transient execution attacks and defenses. In 28th USENIX Security Symposium (USENIX Security 19) (2019), pp. 249-266.
  6. CAULIGI, S., DISSELKOEN, C., GLEISSENTHALL, K. V., TULLSEN, D., STEFAN, D., REZK, T., AND BARTHE, G. Constant-time foun- dations for the new spectre era. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Imple- mentation (2020), pp. 913-926.
  7. CAULIGI, S., DISSELKOEN, C., MOGHIMI, D., BARTHE, G., AND STEFAN, D. Sok: Practical foundations for software spectre defenses. In 2022 IEEE Symposium on Security and Privacy (SP) (2022), IEEE, pp. 666-680.
  8. CHARALAMBOUS, Y., TIHANYI, N., JAIN, R., SUN, Y., FERRAG, M. A., AND CORDEIRO, L. C. A new era in software security: Towards self-healing software via large language models and formal verification. arXiv preprint arXiv:2305.14752 (2023). · arXiv
  9. DOYCHEV, G., K ÖPF, B., MAUBORGNE, L., AND REINEKE, J. Cacheaudit: A tool for the static analysis of cache side channels. ACM Transactions on information and system security (TISSEC) 18, 1 (2015), 1-32.
  10. GARG, S., MOGHADDAM, R. Z., AND SUNDARESAN, N. Rapgen: An approach for fixing code inefficiencies in zero-shot. arXiv preprint arXiv:2306.17077 (2023). · arXiv
  11. GUARNIERI, M., K ÖPF, B., MORALES, J. F., REINEKE, J., AND S ÁNCHEZ, A. Spectector: Principled detection of speculative infor- mation flows. In 2020 IEEE Symposium on Security and Privacy (SP) (2020), IEEE, pp. 1-19.
  12. GUPTA, R., PAL, S., KANADE, A., AND SHEVADE, S. Deepfix: Fixing common c language errors by deep learning. In Proceedings of the aaai conference on artificial intelligence (2017), vol. 31.
  13. JANCAR, J., FOURN É, M., BRAGA, D. D. A., SABT, M., SCHWABE, P., BARTHE, G., FOUQUE, P.-A., AND ACAR, Y. "they're not that hard to mitigate": What cryptographic library developers think about timing attacks. In 2022 IEEE Symposium on Security and Privacy (SP) (2022), IEEE, pp. 632-649.
  14. KANDE, R., PEARCE, H., TAN, B., DOLAN-GAVITT, B., THAKUR, S., KARRI, R., AND RAJENDRAN, J. Llm-assisted generation of hardware assertions. arXiv preprint arXiv:2306.14027 (2023). · arXiv
  15. KOCETKOV, D., LI, R., BEN ALLAL, L., LI, J., MOU, C., MU ÑOZ FERRANDIS, C., JERNITE, Y., MITCHELL, M., HUGHES, S., WOLF, T., BAHDANAU, D., VON WERRA, L., AND DE VRIES, H. The stack: 3 tb of permissively licensed source code. Preprint (2022).
  16. KOCHER, P., HORN, J., FOGH, A., GENKIN, D., GRUSS, D., HAAS, W., HAMBURG, M., LIPP, M., MANGARD, S., PRESCHER, T., ET AL. Spectre attacks: Exploiting speculative execution. In 2019 IEEE Symposium on Security and Privacy (SP) (2019), IEEE, pp. 1- 19.
  17. KOCHER, P. C. Timing attacks on implementations of diffie- hellman, rsa, dss, and other systems. In Advances in Cryptol- ogy-CRYPTO'96: 16th Annual International Cryptology Conference Santa Barbara, California, USA August 18-22, 1996 Proceedings 16 (1996), Springer, pp. 104-113.
  18. LIPP, M., SCHWARZ, M., GRUSS, D., PRESCHER, T., HAAS, W., FOGH, A., HORN, J., MANGARD, S., KOCHER, P., GENKIN, D., YAROM, Y., AND HAMBURG, M. Meltdown: Reading kernel memory from user space. In 27th USENIX Security Symposium (USENIX Security 18) (2018).
  19. MOSIER, N., LACHNITT, H., NEMATI, H., AND TRIPPEL, C. Ax- iomatic hardware-software contracts for security. In Proceedings of the 49th Annual International Symposium on Computer Architecture (New York, NY, USA, 2022), ISCA '22, Association for Computing Machinery, p. 72-86.
  20. OLEKSENKO, O., TRACH, B., SILBERSTEIN, M., AND FETZER, C. Specfuzz: Bringing spectre-type vulnerabilities to the surface. In Proceedings of the 29th USENIX Conference on Security Symposium (USA, 2020), SEC'20, USENIX Association.
  21. PEARCE, H., TAN, B., AHMAD, B., KARRI, R., AND DOLAN- GAVITT, B. Examining Zero-Shot Vulnerability Repair with Large Language Models. In 2023 IEEE Symposium on Security and Privacy (SP) (2023), IEEE.
  22. RODRIGUES, B., QUINT ÃO PEREIRA, F. M., AND ARANHA, D. F. Sparse representation of implicit flows with applications to side- channel detection. In Proceedings of the 25th International Con- ference on Compiler Construction (2016), pp. 110-120.
  23. TARLOW, D., MOITRA, S., RICE, A., CHEN, Z., MANZAGOL, P.- A., SUTTON, C., AND AFTANDILIAN, E. Learning to fix build errors with graph2diff neural networks. In Proceedings of the IEEE/ACM 42nd international conference on software engineering workshops (2020), pp. 19-20.
  24. WANG, G., CHATTOPADHYAY, S., BISWAS, A. K., MITRA, T., AND ROYCHOUDHURY, A. Kleespectre: Detecting information leakage through speculative cache attacks via symbolic execution. ACM Trans. Softw. Eng. Methodol. 29, 3 (jun 2020).
  25. WANG, G., CHATTOPADHYAY, S., GOTOVCHITS, I., MITRA, T., AND ROYCHOUDHURY, A. oo7: Low-overhead defense against spectre attacks via program analysis. IEEE Transactions on Software Engineering (2019).
  26. WANG, S., WANG, P., LIU, X., ZHANG, D., AND WU, D. {CacheD}: Identifying {Cache-Based} timing channels in production software. In 26th USENIX security symposium (USENIX security 17) (2017), pp. 235-252.
  27. WEI, J., WANG, X., SCHUURMANS, D., BOSMA, M., XIA, F., CHI, E., LE, Q. V., ZHOU, D., ET AL. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35 (2022), 24824-24837.
  28. WEISER, S., ZANKL, A., SPREITZER, R., MILLER, K., MANGARD, S., AND SIGL, G. {DATA}-differential address trace analysis: Finding address-based {Side-Channels} in binaries. In 27th USENIX Security Symposium (USENIX Security 18) (2018), pp. 603-620.
  29. WICHELMANN, J., MOGHIMI, A., EISENBARTH, T., AND SUNAR, B. Microwalk: A framework for finding side channels in binaries. In Proceedings of the 34th Annual Computer Security Applications Conference (New York, NY, USA, 2018), ACSAC '18, Association for Computing Machinery, p. 161-173.
  30. WICHELMANN, J., SIECK, F., P ÄTSCHKE, A., AND EISENBARTH, T. Microwalk-ci: practical side-channel analysis for javascript appli- cations. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security (2022), pp. 2915-2929.
  31. WU, M., GUO, S., SCHAUMONT, P., AND WANG, C. Eliminating timing side-channel leaks using program repair. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis (2018), pp. 15-26.
  32. WU, T., JIANG, E., DONSBACH, A., GRAY, J., MOLINA, A., TERRY, M., AND CAI, C. J. Promptchainer: Chaining large language model prompts through visual programming. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (2022), pp. 1-10.
  33. WU, T., TERRY, M., AND CAI, C. J. Ai chains: Transparent and controllable human-ai interaction by chaining large language model prompts. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (2022), pp. 1-22.
  34. WU, Y., JIANG, N., PHAM, H. V., LUTELLIER, T., DAVIS, J., TAN, L., BABKIN, P., AND SHAH, S. How effective are neural networks for fixing security vulnerabilities. arXiv preprint arXiv:2305.18607 (2023). · arXiv
  35. YASUNAGA, M., AND LIANG, P. Break-it-fix-it: Unsupervised learn- ing for program repair. In International Conference on Machine Learning (2021), PMLR, pp. 11941-11952.

Cited by

Sorted by citations

Authors
Tol, M. C., & Sunar, B.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2308.13062
arXiv
2308.13062
OpenAlex
W6959835086
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • National Science Foundation (2026913)
  • Fonds National de la Recherche Luxembourg
  • Qatar National Research Fund
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Cryptographic Implementations and Security
  • Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
  • Physical Sciences › Computer Science › Information Systems › Web Application Security Vulnerabilities
Keywords
side-channel attacks, large language models, software security, leakage mitigation, cost-effective security

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review