R-Judge: Benchmarking Safety Risk Awareness for LLM Agents

The rise and potential of large language model based agents: a survey

Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970 · see citing works

When LLMs meet cybersecurity: a systematic literature review

Zhang, J., Bu, H., Wen, H., Liu, Y., Fei, H., Xi, R., Li, L., Yang, Y., Zhu, H., & Meng, D. (2025) · Cybersecurity · doi:10.1186/s42400-025-00361-w · Cited by 220 · see citing works

AI agents under threat: A survey of key security challenges and future pathways

Deng, Z., Guo, Y., Han, C., Ma, W., Xiong, J., Wen, S., & Xiang, Y. (2025) · ACM Computing Surveys · doi:10.1145/3716628 · Cited by 210 · see citing works

Safeguarding large language models: a survey

Dong, Y., Mu, R., Zhang, Y., Sun, S., Zhang, T., Wu, C., Jin, G., Qi, Y., Hu, J., Meng, J., et al. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11389-2 · Cited by 161 · see citing works

LLM-Based Agents for Tool Learning: A Survey

Xu, W., Huang, C., Gao, S., & Shang, S. (2025) · Data Science and Engineering · doi:10.1007/s41019-025-00296-9 · Cited by 69 · see citing works

Risks of AI scientists: prioritizing safeguarding over autonomy

Tang, X., Jin, Q., Zhu, K., Yuan, T., Zhang, Y., Zhou, W., Qu, M., Zhao, Y., Tang, J., Zhang, Z., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-63913-1 · Cited by 84 · see citing works

Leveraging Large Language Models for Enhancing Safety in Maritime Operations

Miller, T. I., Durlik, I., Kostecka, E., Łobodzińska, A., Łazuga, K., & Kozlovska, P. (2025) · Applied Sciences · doi:10.3390/app15031666 · Cited by 31 · see citing works

Beyond Words: On Large Language Models Actionability in Mission-Critical Risk Analysis

Esposito, M., Palagiano, F., Lenarduzzi, V., & Taibi, D. (2024) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1145/3674805.3695401 · Cited by 22 · see citing works

Position: Prospective of Autonomous Driving - Multimodal LLMs, World Models, Embodied Intelligence, AI Alignment, and Mamba

Ma, Y., Ye, W., Cui, C., Zhang, H., Xing, S., Ke, F., Wang, J., Miao, C., Chen, J., Rezatofighi, H., et al. (2025) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw65960.2025.00114 · Cited by 19 · see citing works

SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigation

Shanmugarasa, Y., Ding, M., Chamikara, M. A. P., & Rakotoarivelo, T. (2025) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3708821.3733888 · Cited by 18 · see citing works

Frontiers of Large Language Model-Based Agentic Systems - Construction, Efficacy and Safety

He, J., Ghosh, R., Walia, K., Chen, J., Dhadiwal, T., Hazel, A., & Inguva, C. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679105 · Cited by 11 · see citing works

Agentic AI in Healthcare: Opportunities, Challenges, and Future Directions

Gridach, M., Nanavati, J., Zine El Abidine, K., Yacoubian, C., & Mack, C. (2026) · ACM Computing Surveys · doi:10.1145/3809164 · Cited by 8 · see citing works

Ensuring Safety in LLM-Driven Robotics: A Cross-Layer Sequence Supervision Mechanism

Wang, Z., Liu, Q., Qin, J., & Li, M. (2024) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros58592.2024.10801576 · Cited by 7 · see citing works

S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Models

Yuan, X., Li, J., Wang, D., Chen, Y., Mao, X., Huang, L., Chen, J., Xue, H., Liu, X., Wang, W., et al. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728971 · Cited by 27 · see citing works

Mitigating Hallucination in Large Language Models: A Capability-Oriented Survey on RAG, Reasoning, and Agentic Systems

Li, Y., Fu, X., Verma, G., Buitelaar, P., & Liu, M. (2025) · arXiv · doi:10.48550/arxiv.2510.24476 · Cited by 4 · see citing works

SafeLawBench: Towards Safe Alignment of Large Language Models

Cao, C., Zhu, H., Ji, J., Sun, Q., Zhu, Z., Yinyu, W., Dai, J., Yang, Y., Han, S., & Guo, Y. (2025) · Findings of the Association for Computational Linguistics: ACL 2025 · doi:10.18653/v1/2025.findings-acl.721 · Cited by 6 · see citing works

Privacy Assessment Methodology for Machine Learning Models and Data Sources

Erdei, R., Pașca, E. M., Delinschi, D., Avram, A., Chereja, I., & Matei, O. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-75010-6_21 · Cited by 4 · see citing works

A Cognitive Architecture for Embodied AI Based on LLM Common-Sense Knowledge

Saladino, A., Brienza, M., Suriani, V., Bloisi, D. D., & Iocchi, L. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-2398-6_23 · Cited by 2 · see citing works

Securing Agents With Tracked Capabilities

Odersky, M., Zhao, Y., Xu, Y., Bračevac, O., & Pham, C. N. (2026) · ACM Conference on AI and Agentic Systems (CAIS) · doi:10.1145/3786335.3813127 · Cited by 2 · see citing works

ASTRAL: Automated Safety Testing of Large Language Models

Ugarte, M., Valle, P., Antonio, J. P., Segura, S., & Arrieta, A. (2025) · doi:10.1109/ast66626.2025.00018 · Cited by 3 · see citing works

SafeToolBench: pioneering a prospective benchmark to evaluating tool utilization safety in LLMs

Xia, H., Wang, H., Liu, Z., Yu, Q., Guo, Y., & Wang, H. (2025) · Findings of the Association for Computational Linguistics: EMNLP 2025 · doi:10.18653/v1/2025.findings-emnlp.958 · Cited by 1

MedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systems

Chen, K., Zhen, T., Wang, H., Liu, K., Li, X., Huo, J., Yang, T., Xu, J., Dong, W., & Gao, Y. (2025) · arXiv · doi:10.48550/arxiv.2505.20824 · Cited by 1 · see citing works

CogTrust: Cognitive Logic-Based framework for dynamic trust evaluation in multi-agent systems

Wang, J., Yang, S., Li, J., Zhao, Y., Wang, Y., & Wu, S. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.131535 · Cited by 1 · see citing works

The CLEF-2026 FinMMEval Lab: Multilingual and Multimodal Evaluation of Financial AI Systems

Xie, Z., Elbadry, R., Zhang, F., Georgiev, G., Peng, X., Qian, L., Huang, J., Dimitrov, D., Jani, V., Dai, Y., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-21321-1_37 · Cited by 1 · see citing works

Characterizing User-Reported Risks across LLM Chatbots

Li, L., Ma, R., Xue, Z., & Xiong, J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791505 · Cited by 1 · see citing works

Zombie Agents: Detecting Semantic Livelock in Long-Horizon Autonomous Software

Khanna, S. (2026) · ACM International Conference on AI-Powered Software (AIware) · doi:10.1145/3805760.3814895 · Cited by 1 · see citing works

Bridging Today and the Future of Humanity: AI Safety in 2024 and BeyondPreprint

Han, S. (2025) · Qeios · doi:10.32388/zhm9uq · Cited by 3 · see citing works

A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents

Huang, W.-C., Zhang, W., Liang, Y., Bei, Y., Chen, Y., Feng, T., Pan, X., Tan, Z., Wang, Y., Wei, T., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.06052

Adversarial Shield: Using Large Language Models to Enhance Response Security Against Adversarial AttacksPreprint

Hamed, M., Kasiem, N., Abdou, M., Aslan, H. K., Khoriba, G. A., & Arafa, T. (2024) · Research Square · doi:10.21203/rs.3.rs-4816315/v1

CRDA: Content Risk Drift Assessment of Large Language Models through Adversarial Multi-Agent Interaction

Liu, Z., Li, G., Shi, B., Zhang, X., Ge, J., Wu, Y., & Lyu, H. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650172 · Cited by 1 · see citing works

CRALA: A Chinese-centric Risk Simulation and Assessment Framework for LLM Agents

Ye, W., Zhang, Z., Li, W., Qin, Y., & Zhang, P. (2024) · International Symposium on Artificial Intelligence and Intelligent Manufacturing (AIIM) · doi:10.1109/aiim64537.2024.10934441

ASTRA: Agentic Steerability and Risk Assessment Framework

Hazan, I., Mathov, Y., Shtar, G., Bitton, R., & Mantin, I. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-32732-1_21 · Cited by 2 · see citing works

Securing Agentic AI with the NIST Cybersecurity Framework 2.0

Metin, B., Karaca, H. S., Iradat, F., & Wynn, M. G. (2025) · National Conference on Electrical and Electronics Engineering (ELECO) · doi:10.1109/eleco69582.2025.11329370

ALARM: Automated MLLM-Based Anomaly Detection in Complex-Environment Monitoring with Uncertainty Quantification

Zhang, C., Lin, R. F., Zhao, X., Guo, P., Li, W., Chen, L., Zhao, C., & Huang, S. (2026) · INFORMS Journal on Data Science · doi:10.1287/ijds.2025.0107

EvalAgent: Interactive Comparative Evaluation of Computer-Using GUI Agents

Yang, Y., Gebreegziabher, S. A., Yoo, H., Chiang, C., Chen, C., Szymanski, A., Do, H. J., Ashktorab, Z., Geyer, W., Gómez-Zará, D., et al. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742414.3795096

Enhancing Risk Awareness in LLM Agents via Probing Safety Boundaries

Jiang, Y., Peng, H.-L., Zheng, Y.-S., Liu, Z.-Y., Lam, K.-Y., & Tan, C.-W. (2026) · IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp55912.2026.11464257

When Harmful Intent Dissolves into Technical Detail: How Safe Are Coding Agents Against Cyber Misuse?

Xu, X., Feng, S., Shen, G., & Zhang, X. (2026) · ACM Conference on AI and Agentic Systems (CAIS) · doi:10.1145/3786335.3813176

A Multi-Agent Framework for Step-Level Tool-Call Safety Evaluation in Industrial Control Systems

Xue, Z., & He, X. (2026) · International Conference on Artificial Intelligence and Industrial Technology Applications (AIITA) · doi:10.1109/aiita69518.2026.11567230

Toward a Principled Framework for Agent Safety Measurement

Lin, S., Suri, A., Oprea, A., & Tan, C. (2026) · ACM SIGOPS Operating Systems Review · doi:10.1145/3830422.3830423

When Prompts Fail to Differentiate: Understanding Discriminative Power in LLM Agent Evaluation

Chen, Z., Wong, W. E., Hsu, C.-W., & Dai, G. (2026) · International Symposium on System Security, Safety, and Reliability (ISSSR) · doi:10.1109/isssr69973.2026.00023

Navigating the Certainty Gap: The Hydraulic Effect of Evidence-Constrained LLMs in Professional Decision-Making

Gundidza, F., Kikuchi, M., & Ozono, T. (2026) · IEEE Access · doi:10.1109/access.2026.3723798

Fix or Break? Exploring Autonomous Remediation of IoT Device Security Misconfigurations Using LLM Agents

Ishihara, E., Takeuchi, A., Fujii, S., Sasaki, T., & Yoshioka, K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-35579-9_2

A Survey of Fuzzing Techniques for Large Language Model Agents: Challenges, Methods, and Future Directions

Cao, Y. (2026) · IEEE International Conference on Big Data and Artificial Intelligence (BDAI) · doi:10.1109/bdai70753.2026.11655176

Evil-AI Benchmark: An Evaluation Framework for Adversarial Security Assessment of LLM Agents in Smart Environments

Varol, A., Shaikh, A., Jalo, H., Muchenje, G., de Cerqueira, J. S., Kemell, K., Motlagh, N. H., Leino, M., Pirkkalainen, H., Abrahamsson, P., et al. (2026) · IEEE Access · doi:10.1109/access.2026.3727159

TOAIAF: A Runtime AI Assurance Control Plane for Trustworthy On-Device Agentic AI in Consumer Intelligence EcosystemsPreprint

Agrawal, R. K., Tambe, W. M. A., & Karra, N. (2026) · Preprints.org · doi:10.20944/preprints202608.2126.v1

Evaluating the Robustness of Safety-Aligned LLM Behavior to Short Contextual PrefixesPreprint

Cholewa, M., & Głomb, P. (2026) · Research Square · doi:10.21203/rs.3.rs-10590385/v1

Effective Segregation of Duties in Agentic Enterprise Authorization: Evaluating Shared-Dependence Risk in Maker–Checker ControlsPreprint

Elmasry, M. A. (2026) · Research Square · doi:10.21203/rs.3.rs-10985030/v1

Formal Runtime Verification for Tool-Using LLM Agents: An Offline Same-Benchmark Study on AgentDojo and STAC

Kekatos, N., Basagiannis, S., Chintri, M., Lekidis, A., Nianios, T., Seitoglou, I., Temperekidis, A., & Katsaros, P. (2026) · ACM Workshop on CPS&IoT Security and Privacy (CPSIoTSec) · doi:10.1145/3847353.3847499

COMPASS: Benchmarking Safety‐Aware Vision‐Language Navigation in Intelligent Transport Environments

Dolgopolyi, R., Khan, M. S., Ahmad, J. E., Vavousis, K., & Pitropakis, N. (2026) · IET Intelligent Transport Systems · doi:10.1049/itr2.70347

LLM-based agents for generalized web automation and schema-guided data extraction: a survey

Khayoub, I., Chadi, M.-A., Mousannif, H., & Mohamed, F. A. (2026) · International Journal of Data Science and Analytics · doi:10.1007/s41060-026-01335-y

Cognitive Autonomous Memory Security (CAMS) against injection and extraction attacks in long-term memory of AI agents

T., D., S., S., Ramamoorthy, A. K., & Balasubramanian, U. M. (2026) · Egyptian Informatics Journal · doi:10.1016/j.eij.2026.100983

SafeAgent: Safeguarding LLM Agents via an Automated Risk Simulator

Zhou, X., Wang, W., Lu, L., Shi, J., Tie, G., Yongtian, X., Chen, L., Zhou, P., Gong, N. Z., & Sun, L. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1501

SafetyQuizzer: Timely and Dynamic Evaluation on the Safety of LLMs

Shi, Z., Jing, S., Cheng, Y., Zhang, H., Wang, Y., Zhang, J., Shen, H., & Cheng, X. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.85

RiskLab: A Controlled Toolkit for Probing Emergent Risks in LLM-Based Multi-Agent Systems

Jiang, Y., Wang, W., Huang, Y., Wang, Y., Zhou, Z., Chen, X., Liu, Y., Chen, P.-Y., Wang, W., & Zhang, X. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-demo.17

Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey

Dong, Z., Zhou, Z., Yang, C., Shao, J., & Qiao, Y. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.375 · Cited by 32 · see citing works

Enhancing Risk Awareness in LLM Agents via Probing Safety Boundaries

Jiang, Y., Peng, H.-L., Zheng, Y.-S., Liu, Z.-Y., Lam, K.-Y., & Tan, C.-W. (2026) · IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp55912.2026.11464257