PLeak: Prompt Leaking Attacks against Large Language Model Applications
Open accessAbstract
Large Language Models (LLMs) enable a new ecosystem with many downstream applications, called LLM applications, with different natural language processing tasks. The functionality and performance of an LLM application highly depend on its system prompt, which instructs the backend LLM on what task to perform. Therefore, an LLM application developer often keeps a system prompt confidential to protect its intellectual property. As a result, a natural attack, called prompt leaking, is to steal the system prompt from an LLM application, which compromises the developer's intellectual property. Existing prompt leaking attacks primarily rely on manually crafted queries, and thus achieve limited effectiveness.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLMs can be sensitive to irrelevant context, exhibiting context leakage.
“Prior work has shown that LLMs can be sensitive to irrelevant context, exhibiting context leakage (Mireshghallah et al., 2023; Gupta et al., 2024; Hui et al., 2024), and display sycophantic behavior with responses favoring perceived user preferences rather than objective evidence (Sharma et al., 2023; Perez et al., 2023; Fanous et al., 2025).”
From PersistBench: When Should Long-Term Memories Be Forgotten by LLMs? (2026) · cites this paper as Hui et al., 2024
PersistBench covers a wider and different set of risks (cross-domain leakage and sycophancy) caused by long-term user memories than prior work that targets privacy-centric failures or risks confined to a single short context window.
“Unlike prior work that primarily targets privacy-centric failures such as PII disclosure and contextual integrity violations (Mireshghallah et al., 2025), or risks confined to a single short context window (Hui et al., 2024), PersistBench covers a wider and different set of risks (cross-domain leakage and sycophancy) caused by long-term user memories.”
From PersistBench: When Should Long-Term Memories Be Forgotten by LLMs? (2026) · cites this paper as Hui et al., 2024
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.243 · Cited by 3,096 · see citing works
The Power of Scale for Parameter-Efficient Prompt Tuning
Lester, B., Al-Rfou, R., & Constant, N. (2021) · arXiv · doi:10.48550/arxiv.2104.08691 · Cited by 3,092
Universal Adversarial Perturbations
Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., & Frossard, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.17 · Cited by 2,680 · see citing works
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604 · see citing works
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013 · see citing works
Language Models as Knowledge Bases?
Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1250 · Cited by 1,834 · see citing works
Making Pre-trained Language Models Better Few-shot Learners
Gao, T., Fisch, A., & Chen, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.295 · Cited by 1,380 · see citing works
The Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304 · see citing works
AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., & Singh, S. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.346 · Cited by 1,279 · see citing works
Association for Computational Linguistics
Hirst, G. (2006) · Elsevier eBooks · doi:10.1016/b0-08-044854-2/05234-2 · Cited by 1,152 · see citing works
Liu, X., Zheng, Y., Du, Z., Ding, M., Qian, Y., Yang, Z., & Tang, J. (2023) · AI Open · doi:10.1016/j.aiopen.2023.08.012 · Cited by 1,124 · see citing works
Deep reinforcement learning from human preferences
Christiano, P., Leike, J., Brown, T. B., Martic, M., Legg, S., & Amodei, D. (2017) · arXiv · doi:10.48550/arxiv.1706.03741 · Cited by 1,058 · see citing works
HotFlip: White-Box Adversarial Examples for Text Classification
Ebrahimi, J., Rao, A., Lowd, D., & Dou, D. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-2006 · Cited by 948 · see citing works
Baseline Defenses for Adversarial Attacks Against Aligned Language ModelsPreprint
Jain, N., Schwarzschild, A., Wen, Y., Somepalli, G., Kirchenbauer, J., Chiang, P.-Y., Goldblum, M., Saha, A., Geiping, J., & Goldstein, T. (2023) · arXiv.org · doi:10.48550/arxiv.2309.00614 · Cited by 838 · see citing works
P-Tuning: Prompt Tuning Can Be Comparable to Fine-tuning Across Scales and Tasks
Liu, X., Ji, K., Fu, Y., Tam, W., Du, Z., Yang, Z., & Tang, J. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/2022.acl-short.8 · Cited by 810 · see citing works
Stealing Machine Learning Models via Prediction APIs
Tramèr, F., Zhang, F., Juels, A., Reiter, M. K., & Ristenpart, T. (2016) · arXiv · doi:10.48550/arxiv.1609.02943 · Cited by 796 · see citing works
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.759 · Cited by 755 · see citing works
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?
Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., & Zettlemoyer, L. (2022) · arXiv · doi:10.48550/arxiv.2202.12837 · Cited by 752 · see citing works
Pang, B., & Lee, L. (2005) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0506075 · Cited by 744 · see citing works
Extracting Training Data from Large Language Models
Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07805 · Cited by 727 · see citing works
Universal Adversarial Triggers for Attacking and Analyzing NLP
Wallace, E., Feng, S., Kandpal, N., Gardner, M., & Singh, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1221 · Cited by 642 · see citing works
Multitask Prompted Training Enables Zero-Shot Task Generalization
Sanh, V., Webson, A., Raffel, C., Bach, S. H., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Scao, T. L., Raja, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.08207 · Cited by 612 · see citing works
Social IQa: Commonsense Reasoning about Social Interactions
Sap, M., Rashkin, H., Chen, D., LeBras, R., & Choi, Y. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1454 · Cited by 588 · see citing works
Good debt or bad debt: Detecting semantic orientations in economic texts
Malo, P., Sinha, A., Korhonen, P., Wallenius, J., & Takala, P. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23062 · Cited by 513 · see citing works
Calibrate Before Use: Improving Few-Shot Performance of Language Models
Zhao, T. Z., Wallace, E., Feng, S., Klein, D., & Singh, S. (2021) · arXiv · doi:10.48550/arxiv.2102.09690 · Cited by 513 · see citing works
Stealing Hyperparameters in Machine Learning
Wang, B., & Gong, N. Z. (2018) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2018.00038 · Cited by 491 · see citing works
Self-citationOpt: Open pre-trained transformer language models
Zhang, S. (2025) · TIB Data Manager · doi:10.57702/dn0pmcbs · Cited by 405 · see citing works
Beam Search Strategies for Neural Machine Translation
Freitag, M., & Al-Onaizan, Y. (2017) · doi:10.18653/v1/w17-3207 · Cited by 394 · see citing works
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Cui, C., Bousquet, O., Le, Q., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.10625 · Cited by 380 · see citing works
Multi-step Jailbreaking Privacy Attacks on ChatGPT
Li, H., Guo, D., Fan, W., Xu, M., Huang, J., Meng, F., & Song, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.272 · Cited by 269 · see citing works
Universal and Transferable Adversarial Attacks on Aligned Language Models
Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z., & Fredrikson, M. (2023) · arXiv · doi:10.48550/arxiv.2307.15043 · Cited by 197 · see citing works
How Can We Accelerate Progress Towards Human-like Linguistic Generalization?
Linzen, T. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.465 · Cited by 148 · see citing works
An Explanation of In-context Learning as Implicit Bayesian Inference
Xie, S. M., Raghunathan, A., Liang, P., & Ma, T. (2021) · arXiv · doi:10.48550/arxiv.2111.02080 · Cited by 138 · see citing works
Making Pre-trained Language Models Better Few-shot Learners
Gao, T., Fisch, A., & Chen, D. (2020) · arXiv · doi:10.48550/arxiv.2012.15723 · Cited by 124 · see citing works
Ignore Previous Prompt: Attack Techniques For Language Models
Perez, F., & Ribeiro, I. (2022) · arXiv · doi:10.48550/arxiv.2211.09527 · Cited by 110 · see citing works
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., Freedman, R., Korbak, T., Lindner, D., Freire, P., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.15217 · Cited by 101 · see citing works
Stealing Links from Graph Neural NetworksPreprint
He, X., Jia, J., Michael Backes, A., Gong, N. Z., & Zhang, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.02131 · Cited by 94 · see citing works
Self-citationInducing anxiety in large language models increases exploration and biasPreprint
Coda-Forno, J., Witte, K., Jagadish, A. K., Binz, M., Akata, Z., & Schulz, E. (2023) · arXiv.org · doi:10.48550/arxiv.2304.11111 · Cited by 79 · see citing works
In ChatGPT We Trust? Measuring and Characterizing the Reliability of ChatGPT
Shen, X., Chen, Z., Backes, M., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2304.08979 · Cited by 76 · cited 2× · see citing works
“Shen et al. [46], and Li et al. [56] exploit indirect prompts to induce LaMDA to elicit erroneous responses.”
Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!Preprint
Qi, X., Zeng, Y., Xie, T., Chen, P., Jia, R., Mittal, P., & Henderson, P. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.03693 · Cited by 57 · see citing works
AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language ModelsPreprint
Liu, X., Xu, N., Chen, M., & Xiao, C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.04451 · Cited by 41 · see citing works
EED: Extended Edit Distance Measure for Machine Translation
Stanchev, P., Wang, W., & Ney, H. (2019) · Conference on Machine Translation (WMT) · doi:10.18653/v1/w19-5359 · Cited by 31 · see citing works
Stealing the Decoding Algorithms of Language Models
Naseh, A., Krishna, K., Iyyer, M., & Houmansadr, A. (2023) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3576915.3616652 · Cited by 17 · see citing works
Formalizing and Benchmarking Prompt Injection Attacks and Defenses
Liu, Y., Jia, Y., Geng, R., Jia, J., & Gong, N. Z. (2023) · arXiv · doi:10.48550/arxiv.2310.12815 · Cited by 14 · see citing works
Self-citationWhat Do You Get When You Cross Beam Search with Nucleus Sampling?
Shaham, U., & Levy, O. (2022) · Workshop on Insights from Negative Results in NLP (Insights) · doi:10.18653/v1/2022.insights-1.5 · Cited by 8 · see citing works
Effective Prompt Extraction from Language Models
Zhang, Y., Carlini, N., & Ippolito, D. (2023) · arXiv · doi:10.48550/arxiv.2307.06865 · Cited by 8 · see citing works
He, X., Zannettou, S., Shen, Y., & Zhang, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.05596 · Cited by 7 · see citing works
OPT: Open Pre-trained Transformer Language Models
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.01068
Flagged first, then by citations
Cited by
Sorted by citations
Wang, Y., Pan, Y., Su, Z., Deng, Y., Zhao, Q., Du, L., Luan, T. H., Kang, J., & Niyato, D. T. (2025) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2025.3576176 · Cited by 86 · see citing works
Optimization-based Prompt Injection Attack to LLM-as-a-Judge
Shi, J., Yuan, Z., Liu, Y., Huang, Y., Zhou, P., Sun, L., & Gong, N. Z. (2024) · Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3658644.3690291 · Cited by 70 · see citing works
Self-citationBrohi, S., Mastoi, Q.-U.-A., Jhanjhi, N. Z., & Pillai, T. R. (2025) · Algorithms · doi:10.3390/a18080499 · Cited by 69 · see citing works
Unique Security and Privacy Threats of Large Language Models: A Comprehensive Survey
Wang, S., Zhu, T., Liu, B., Ding, M., Ye, D., Zhou, W., & Yu, P. S. (2025) · ACM Computing Surveys · doi:10.1145/3764113 · Cited by 64 · see citing works
Security concerns for Large Language Models: A survey
Li, M. Q., & Fung, B. C. M. (2025) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2025.104284 · Cited by 26 · see citing works
DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks
Liu, Y., Jia, Y.-Q., Jia, J.-Y., Song, D., & Gong, N. (2025) · IEEE Symposium on Security and Privacy · doi:10.1109/sp61157.2025.00250 · Cited by 203 · see citing works
Self-citationUnveiling Privacy Risks in LLM Agent Memory
Wang, B., He, W., Zeng, S., Xiang, Z., Xing, Y., Tang, J., & He, P. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.1227 · Cited by 24 · see citing works
A Survey on Model Extraction Attacks and Defenses for Large Language Models
Zhao, K., Li, L., Ding, K., Gong, N. Z., Zhao, Y., & Dong, Y. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3736573 · Cited by 24 · see citing works
Self-citationA Review of Prompt Engineering Techniques for Large Language Models
Naser, M. Z. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2607553 · Cited by 17 · see citing works
On the (In)Security of LLM App Stores
Hou, X., Zhao, Y., & Wang, H. (2025) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp61157.2025.00117 · Cited by 23 · see citing works
A comprehensive introspection on AI risks: taxonomy, challenges, and future directions
Ray, P. P. (2026) · Iran Journal of Computer Science · doi:10.1007/s42044-025-00370-5 · Cited by 13 · see citing works
A Survey on the Recent Advancements in Human-Centered Dialog Systems
Oruche, R., Goruganthu, S. K., Akula, R., Cheng, X., Goni, M. A., Shibo, B. W., Kee, K. F., Zampieri, M., & Calyam, P. P. (2025) · ACM Computing Surveys · doi:10.1145/3729220 · Cited by 9 · see citing works
The Early Bird Catches the Leak: Unveiling Timing Side Channels in LLM Serving Systems
Song, L., Pang, Z., Wang, W., Wang, Z., Wang, X., Chen, H., Song, W., Jin, Y., Meng, D., & Hou, R. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3622954 · Cited by 7 · see citing works
Vulnerability Detection in Large Language Models: Addressing Security Concerns
Yaala, S. B., & Bouallègue, R. (2025) · Journal of Cybersecurity and Privacy · doi:10.3390/jcp5030071 · Cited by 6 · see citing works
Prompt-Based Jailbreaking of Leading LLM Chatbots: A Survey of Attacks and Defenses
Knowlton, B., Campa, J., Gallo, D. S., Dajani, K. F., & Alzahrani, N. (2026) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2026.3665656 · Cited by 4 · see citing works
Riddle Me This! Stealthy Membership Inference for Retrieval-Augmented Generation
Naseh, A., Peng, Y., Suri, A., Chaudhari, H., Oprea, A., & Houmansadr, A. (2025) · Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3719027.3744840 · Cited by 3 · see citing works
LLM-driven bot infiltration: protecting web surveys through prompt injections
Höhne, J. K., Claassen, J., & Wolf, B. L. (2025) · International Journal of Social Research Methodology · doi:10.1080/13645579.2025.2598606 · Cited by 3 · see citing works
Multi-Agent AI Framework for Threat Mitigation and Resilience in Machine Learning Systems
Foundjem, A. T., Tidjon, L. N., Silva, L. D., & Khomh, F. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3780095 · Cited by 3 · see citing works
A Critical Evaluation of Defenses against Prompt Injection Attacks: [Dataset/Tool Paper]
Jia, Y., Shao, Z., Liu, Y., Jia, J., Song, D. X., & Gong, N. Z. (2026) · ACM Symposium on Access Control Models and Technologies (SACMAT) · doi:10.1145/3750555.3811884 · Cited by 11 · see citing works
Self-citationRAGLeak: Membership Inference Attacks on RAG-Based Large Language Models
Feng, K., Zhang, G., Tian, H., Xu, H., Zhang, Y., Zhu, T., Ding, M., & Liu, B. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-9101-2_8 · Cited by 3 · see citing works
Paim, K. O., Mansilha, R. B., Kreutz, D., Franco, M. F., & Cordeiro, W. L. D. C. (2025) · Simpósio Brasileiro de Cibersegurança (SBSeg) · doi:10.5753/sbseg.2025.11448 · Cited by 3 · see citing works
Reconstruction of Differentially Private Text Sanitization via Large Language Models
Pang, S., Lu, Z., Wang, H., Fu, P., Zhou, Y., & Xue, M. (2025) · International Symposium on Research in Attacks, Intrusions and Defenses (RAID) · doi:10.1109/raid67961.2025.00022 · Cited by 3 · see citing works
A Systematic Literature Review on SWOT Analysis of Prompt Engineering Techniques
Singh, A., Chatta, N. K., Ehtesham, A., Kumar, S., Gupta, G. K., & Khoei, T. T. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3626467 · Cited by 2 · see citing works
AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agents
Hu, H., Chen, P., Zhao, Y., & Chen, Y. (2025) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3719027.3765064 · Cited by 6 · see citing works
How Not to Detect Prompt Injections with an LLM
Choudhary, S., Anshumaan, D., Palumbo, N., & Jha, S. (2025) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3733799.3762980 · Cited by 2 · see citing works
Simulation of prompt injection attacks on generative pre-trained transformers models
Kurniawan, A. F., & Chandra, M. B. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.08.292 · Cited by 1 · see citing works
Unsafe by Design? A First Look at Security and Privacy Risks in OpenAI’s Custom GPT Ecosystem
Ogundoyin, S. O., Ikram, M., Asghar, H. J., Zhao, B. Z. H., & Kâafar, M. A. (2025) · Workshop on Privacy in the Electronic Society · doi:10.1145/3733802.3764054 · Cited by 1 · see citing works
PromptFuzz: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMs
Shao, Y., Yu, J., Miao, H., Gou, G., Li, Z., & Shi, J. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3666893 · Cited by 1 · see citing works
Mitigating Prompt Injection Attacks in ModelAgnostic Networks (MAN)
Mishra, A., Preet, S., Gupta, B. B., Rawat, S. S., Arya, V., & Katiyar, V. (2025) · International Conference on Internet of Things: Smart Innovation and Usages (IoT-SIU) · doi:10.1109/iot-siu65919.2025.11402845 · Cited by 1 · see citing works
Li, J., Wang, J., Zhao, Z., & Wei, Z. (2026) · Computer Standards & Interfaces · doi:10.1016/j.csi.2026.104181 · Cited by 1 · see citing works
PersistBench: When Should Long-Term Memories Be Forgotten by LLMs?
Pulipaka, S., Chen, O., Sharma, M., Bajwa, T. S., Raina, V., & Sheth, I. (2026) · arXiv · doi:10.48550/arxiv.2602.01146
You Can't Steal Nothing: Mitigating Prompt Leakages in LLMs via System Vectors
Cao, B., Li, C., Cao, Y., Ge, Y., Wang, T., & Chen, J. (2025) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3719027.3765124 · Cited by 1 · see citing works
Self-citationWang, Y.-L., Wu, J., Jiang, T.-Y., Liu, M.-W., Chen, J.-C., Wang, C., Shi, E.-S., Liu, X.-L., Ma, Y.-C., & Zheng, Z.-B. (2025) · International Conference on Automated Software Engineering · doi:10.1109/ase63991.2025.00070 · Cited by 4 · see citing works
Uncovering Prompt Elements: Cloning System Prompts from Behavioral Traces
Qian, Y., Peng, F., Wu, H., Chen, L., & Mao, B. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00045
Critical Safety Attention Heads: Architecture-Dependent Vulnerabilities in LLMsPreprint
Sha, L., Sun, P., Xue, H., Hao, S., & Xiao, F. (2026) · Research Square · doi:10.21203/rs.3.rs-8886237/v1
Prompt Attacks and Safeguards in Large Language Models: A Survey
Mojallal, K., Sadr, P., Ahmadian, S., & Alhadidi, D. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-7078-2_29
Towards Graph-Based Detection of Jailbreak and Prompt-Leakage Attacks in LLMs
Forough, J., Maheri, M. M., & Haddadi, H. (2026) · Workshop on Machine Learning and Systems (EuroMLSys) · doi:10.1145/3805621.3807629
Sami, A. A., Debnath, G., Dey, R., & Chowdhury, A. N. (2025) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit68739.2025.11491067
Machine Learning and Hardware Security: The Role of AI for Hardware in the Security Era
Deng, S., Xu, Y., Li, M., Jin, Y., & Wang, J. (2026) · Springer eBooks · doi:10.1007/978-3-032-17948-7_12
Mitigating Security Risks in Large Language Models: A Full Lifecycle Perspective
Wang, Y. K., Bai, Z., Huo, J., Wang, B., Liu, J., Cao, H., Meng, F., Yang, X., & Gao, Y. (2026) · Machine Learning · doi:10.1007/s10994-025-06930-x
Sahoo, P. S., & Mishra, M. (2026) · International Conference on Air Defense and Security (ICADS) · doi:10.1109/icads69450.2026.11545491
Demystifying LLM API Misuses: A Lifecycle-Based Empirical Study on Real-World Android Apps
Wen, J., & Zhao, Q. (2026) · ACM Conference on Data and Application Security and Privacy (CODASPY) · doi:10.1145/3800506.3803496
LLMThief: Evaluating Configuration Leaking Risks in Commercial LLM App Stores
Chen, P., Jiang, J., Chen, J., Qin, F., Zhang, M., Zhang, J., Duan, H., Shen, K., & Jiang, H. (2026) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp63933.2026.00195
PromptCOS: Towards Content-Only System Prompt Copyright Auditing for LLMs
Yang, Y., Li, Y., Yao, H., Huang, E., Shao, S., Wang, Y., Wang, Z., Tao, D., & Qin, Z. (2026) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp63933.2026.00142
Roadmap and benchmarking: Privacy in federated load forecasting
Abebe, W. M., Potnis, A. V., Sadik, J., Cunningham, J., Longcoy, A., Prout, R., Sundararajan, A., Chinthavali, S., & Lunga, D. (2026) · Future Generation Computer Systems · doi:10.1016/j.future.2026.108713
Magazov, R., Abeshev, K., Shamuratov, Y., Uralova, F., & Aksholak, G. (2026) · AI · doi:10.3390/ai7080287
Privacy Risk Assessment of AI Models in the Healthcare Domain
Anderson, M., Goldsteen, A., Amit, G., Shachor, S., Razinkov, N., Koren, N., & Farkash, A. (2026) · Springer eBooks · doi:10.1007/978-3-032-25917-2_7
A Comparative Survey of Security Risks in AI Systems: From LLMs to AI Agents and Embodied Agents
Wu, B., Li, Q., Zhou, C., Wang, T., & Ji, S. (2026) · ACM Computing Surveys · doi:10.1145/3837083
A Comparative Analysis of Security Vulnerabilities and Defense Mechanisms in Large Language Models
Barek, A., Rashid, M. B., Riad, A. K. I., Francia, G., Cetinkaya, C., Shahriar, H., Khan, M. U., & Ahamed, S. I. (2026) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac69091.2026.00461
Ethical Implications of Prompt Injection in AI-mediated Grading: An Adversarial Red-Team Evaluation
Humble, N. (2026) · Journal of Academic Ethics · doi:10.1007/s10805-026-09758-9
Data security in large language models: risks, defense, and directions
Chen, K., Zhou, X., Yu, Y., Lin, Y., Chen, H., Cai, C., & Shen, L. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-01200-9 · Cited by 2 · see citing works
Narváez, A. R., Martinez, M. L. B., Puente, J. B., & Molina, H. B. (2026) · Electronics · doi:10.3390/electronics15194416
RegexStealer: A Directed-Acyclic-Graph-Based Regex Stealing Against Web Application Firewalls
Li, D., Ling, X., Chen, X., Wu, C., Liu, X., & Khan, M. K. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3737638
Function Calling as a Flexible LLM Defense Add-On: Capability and Application Exploration
Ji, Z., Wu, D., Wang, W., Ma, P., Wang, S., Ma, L., & Rahmel, J. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832209
Prompt Injection Threats in Azure-Based Large Language Model Applications
Lakavath, S. R. (2026) · Journal of Computer Science and Information Technology · doi:10.61424/jcsit.v3i2.1077
Introduction to AI Security: Understanding Prompt Injection Attack
Tikekar, P. C., Gawande, A. R., & Bhad, P. A. (2026) · International Conference on Computer, Mechatronics, Robotics & Artificial Intelligence (ICCMRAI) · doi:10.1109/iccmrai67732.2026.11723765
Yi, Z., Yang, Z., Liu, Q., Ni, R., Han, S., Li, C., Zhang, Z., Xu, P., & Ruan, W. (2026) · CCF Transactions on Pervasive Computing and Interaction · doi:10.1007/s42486-026-00252-0
PAIR: Against Inference-Time Attacks on Fine-Tuned LLMs via Path-Level Inference Randomization
Li, J., Zhou, W., Ye, D., Huo, H., Yu, X., Wang, S., & Zhou, W. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3744609
Prompt Injection Attack to Tool Selection in LLM Agents
Shi, J., Yuan, Z., Tie, G., Zhou, P., Gong, N. Z., & Sun, L. (2025) · arXiv · doi:10.48550/arxiv.2504.19793 · Cited by 2 · see citing works
Self-citation
- Authors
- Hui, B., Yuan, H., Gong, N., Burlina, P., & Cao, Y.
- Affiliations
- Johns Hopkins University (US)
- Duke University (US)
- Johns Hopkins University Applied Physics Laboratory (US)
- Venue
- Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security
- Publisher
- ACM
- Published
- 2024-12-02
- Pages
- 3600-3614
- ISSN
- 1543-7221 (ISSN-L)
- OpenAlex
- W4405181744
- Cited by
- 55
- References
- 47
- Citations by year
- 2026: 28 · 2025: 26 · 2024: 1
- FWCI
- 15.88
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM SIGSAC Conference on Computer and Communications Security (CCS) (publishedVersion, cc-by-nc-sa)
- Funders
- Army Research Office (ARO) (W911NF2110182)
- National Science Foundation (NSF) (OAC-23-19742, CNS-21-31859, CNS- 21-12562, CNS-19-37786, CNS-21-25977, and CNS-19-37787)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- intellectual property protection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
