Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 1

The DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.

PEAR: Planner-Executor Agent Robustness Benchmark

Dong, Shen, Zhang, Mingxuan, He, Pengfei, Ma, Li, Thuraisingham, Bhavani, Liu, Hui, Xing, Yue·2026·Findings of the Association for Computational Linguistics: EACL 2026
DOI 10.18653/v1/2026.findings-eacl.237
Open this paper in CoChat — free account

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2026.findings-eacl.237

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 An overview of PEAR . The center panel shows the planner-executor MAS alongside example user tasks. The left shows the attack tasks; red arrows mark where these attacks are injected into MAS components. The right demonstrates the evaluation metrics used in our study.
Figure 1 An overview of PEAR . The center panel shows the planner-executor MAS alongside example user tasks. The left shows the attack tasks; red arrows mark where these attacks are injected into MAS components. The right demonstrates the evaluation metrics used in our study.CC-BY · From the PDF
Figure 2 Utility comparison across different planner-executor configurations for the Claude, Deepseek, GPT, and Gemini families. Each bar corresponds to a specific combination of planner/executor pairs, as indicated on the X-axis. Each bar represents a planner-executor pair, with height showing the mean utility and error bars indicating the standard deviation across five runs. Utility is computed per environment and then macro-averaged across environments. Detailed numerical results are reported in Table 10 in Appendix E.
Figure 2 Utility comparison across different planner-executor configurations for the Claude, Deepseek, GPT, and Gemini families. Each bar corresponds to a specific combination of planner/executor pairs, as indicated on the X-axis. Each bar represents a planner-executor pair, with height showing the mean utility and error bars indicating the standard deviation across five runs. Utility is computed per environment and then macro-averaged across environments. Detailed numerical results are reported in Table 10 in Appendix E.CC-BY · From the PDF
Figure 3 ASR versus utility comparison under harmful and privacy tasks with error bar included for utility (horizontal) and ASR (vertical).
Figure 3 ASR versus utility comparison under harmful and privacy tasks with error bar included for utility (horizontal) and ASR (vertical).CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 8 references

Research works printed but not linked: 49

  1. Tamer Abuelsaad, Deepak Akkil, Prasenjit Dey, Ashish Jagmohan, Aditya Vempaty, and Ravi Kokku. 2024. Agent-e: From autonomous web navigation to foun- dational design principles in agentic systems. arXiv preprint arXiv:2407.13032. · arXiv
  2. Alfonso Amayuelas, Xianjun Yang, Antonis Antoniades, Wenyue Hua, Liangming Pan, and William Wang. 2024. Multiagent collaboration attack: Investigating adversarial attacks in large language model collabo- rations via debate. arXiv preprint arXiv:2406.14711. · arXiv
  3. Mert Cemri, Melissa Z Pan, Shuyi Yang, Lakshya A Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kan- nan Ramchandran, and 1 others. 2025. Why do multi-agent llm systems fail? arXiv preprint arXiv:2503.13657. · arXiv
  4. Kai Chen, Xinfeng Li, Tianpei Yang, Hewei Wang, Wei Dong, and Yang Gao. 2025. Medsentry: Understand- ing and mitigating safety risks in medical llm multi- agent systems. arXiv preprint arXiv:2505.20824. · arXiv
  5. Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, and 1 others. 2024a. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors. In ICLR.
  6. Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. 2024b. Agentpoison: Red-teaming llm agents via poisoning memory or knowledge bases. Advances in Neural Information Processing Systems, 37:130185-130213.
  7. Edoardo Debenedetti, Jie Zhang, Mislav Balunović, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. 2024. Agentdojo: A dynamic environment to evaluate attacks and defenses for llm agents. arXiv e-prints, pages arXiv-2406.
  8. Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jil- iang Tang, Tianming Liu, Hui Liu, and Zhen Xiang. 2025. A practical memory injection attack against llm agents. arXiv preprint arXiv:2503.03704. · arXiv
  9. Lutfi Eren Erdogan, Nicholas Lee, Sehoon Kim, Suhong Moon, Hiroki Furuta, Gopala Anumanchipalli, Kurt Keutzer, and Amir Gholami. 2025. Plan-and-act: Improving planning of agents for long-horizon tasks. arXiv preprint arXiv:2503.09572. · arXiv
  10. Chengquan Guo, Xun Liu, Chulin Xie, Andy Zhou, Yi Zeng, Zinan Lin, Dawn Song, and Bo Li. 2024a. Redcode: Risky code execution and generation benchmark for code agents. Advances in Neural In- formation Processing Systems, 37:106190-106236.
  11. Pengfei He, Yuping Lin, Shen Dong, Han Xu, Yue Xing, and Hui Liu. 2025a. Red-teaming llm multi-agent systems via communication attacks. In Findings of the Association for Computational Linguistics: ACL 2025, pages 6726-6747.
  12. Pengfei He, Yue Xing, Han Xu, Zhen Xiang, and Jiliang Tang. 2025b. Multi-faceted studies on data poison- ing can advance llm development. arXiv preprint arXiv:2502.14182. · arXiv
  13. Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, and 1 others. 2024. Metagpt: Meta programming for a multi-agent collaborative framework. International Conference on Learning Representations, ICLR.
  14. Jen-tse Huang, Jiaxu Zhou, Tailin Jin, Xuhui Zhou, Zixi Chen, Wenxuan Wang, Youliang Yuan, Michael R Lyu, and Maarten Sap. 2024. On the resilience of llm- based multi-agent collaboration with faulty agents. arXiv preprint arXiv:2408.00989. · arXiv
  15. Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. Camel: Communicative agents for" mind" exploration of large language model society. Advances in Neural Information Processing Systems, 36:51991-52008.
  16. Xinfeng Li, Tianze Qiu, Yingbin Jin, Lixu Wang, Han- qing Guo, Xiaojun Jia, Xiaofeng Wang, and Wei Dong. 2026. Webcloak: Characterizing and mitigat- ing the threats of llm-driven web agents as intelligent scrapers. In Proceedings of the 2026 IEEE Sympo- sium on Security and Privacy (S&P).
  17. Hanjun Luo, Shenyu Dai, Chiming Ni, Xinfeng Li, Guibin Zhang, Kun Wang, Tongliang Liu, and Hanan Salam. 2025. Agentauditor: Human-level safety and security evaluation for llm agents. In Advances in Neural Information Processing Systems.
  18. Zhao Mandi, Shreeya Jain, and Shuran Song. 2024. Roco: Dialectic multi-robot collaboration with large language models. In 2024 IEEE International Con- ference on Robotics and Automation (ICRA), pages 286-299. IEEE.
  19. Fábio Perez and Ian Ribeiro. 2022. Ignore previous prompt: Attack techniques for language models. arXiv preprint arXiv:2211.09527. · arXiv
  20. Archiki Prasad, Alexander Koller, Mareike Hartmann, Peter Clark, Ashish Sabharwal, Mohit Bansal, and Tushar Khot. 2023. Adapt: As-needed decompo- sition and planning with language models. arXiv preprint arXiv:2311.05772. · arXiv
  21. Yangjun Ruan, Honghua Dong, Andrew Wang, Sil- viu Pitis, Yongchao Zhou, Jimmy Ba, Yann Dubois, Chris J Maddison, and Tatsunori Hashimoto. 2023. Identifying the risks of lm agents with an lm- emulated sandbox. arXiv preprint arXiv:2309.15817. · arXiv
  22. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettle- moyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Pro- cessing Systems, 36:68539-68551.
  23. Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Michael Moor, Zicheng Liu, and Emad Barsoum. 2025. Agent laboratory: Using llm agents as research assis- tants. arXiv preprint arXiv:2501.04227. · arXiv
  24. Marc Schmitt and Ivan Flechais. 2024. Digital decep- tion: Generative artificial intelligence in social engi- neering and phishing. Artificial Intelligence Review, 57(12):1-23.
  25. Chenyang Shao, Xinyuan Hu, Yutang Lin, and Fengli Xu. 2025. Division-of-thoughts: Harnessing hy- brid language model synergy for efficient on-device agents. In Proceedings of the ACM on Web Confer- ence 2025, pages 1822-1833.
  26. Yijia Shao, Tianshi Li, Weiyan Shi, Yanchen Liu, and Diyi Yang. 2024. Privacylens: Evaluating privacy norm awareness of language models in action. Ad- vances in Neural Information Processing Systems, 37:89373-89407.
  27. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023. Hugging- gpt: Solving ai tasks with chatgpt and its friends in hugging face. Advances in Neural Information Processing Systems, 36:38154-38180.
  28. Harsh Singh, Rocktim Jyoti Das, Mingfei Han, Preslav Nakov, and Ivan Laptev. 2024. Malmm: Multi-agent large language models for zero-shot robotics manipu- lation. arXiv preprint arXiv:2411.17636. · arXiv
  29. Yashar Talebirad and Amirhossein Nadiri. 2023. Multi- agent collaboration: Harnessing the power of intelli- gent llm agents. arXiv preprint arXiv:2306.03314. · arXiv
  30. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Man- dlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291. · arXiv
  31. Xiaoqiang Wang and Bang Liu. 2024. Oscar: Oper- ating system control via state-aware reasoning and re-planning. arXiv preprint arXiv:2410.18963. · arXiv
  32. Qianshan Wei, Tengchao Yang, Yaochen Wang, Xin- feng Li, Lijun Li, Zhenfei Yin, Yi Zhan, Thorsten Holz, Zhiqiang Lin, and XiaoFeng Wang. 2025. A-memguard: A proactive defense framework for llm-based agent memory. arXiv preprint arXiv:2510.02373. · arXiv
  33. Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. 2023. Auto- gen: Enabling next-gen llm applications via multi- agent conversation framework. arXiv preprint arXiv:2308.08155. · arXiv
  34. Zhiyong Wu, Chengcheng Han, Zichen Ding, Zhenmin Weng, Zhoumianze Liu, Shunyu Yao, Tao Yu, and Lingpeng Kong. 2024. Os-copilot: Towards gener- alist computer agents with self-improvement. arXiv preprint arXiv:2402.07456. · arXiv
  35. Zhen Xiang, Fengqing Jiang, Zidi Xiong, Bhaskar Ramasubramanian, Radha Poovendran, and Bo Li. 2024. Badchain: Backdoor chain-of-thought prompt- ing for large language models. arXiv preprint arXiv:2401.12242. · arXiv
  36. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR).
  37. Miao Yu, Shilong Wang, Guibin Zhang, Junyuan Mao, Chenlong Yin, Qijiong Liu, Qingsong Wen, Kun Wang, and Yang Wang. 2024. Netsafe: Exploring the topological safety of multi-agent networks. arXiv preprint arXiv:2410.15686. · arXiv
  38. Tongxin Yuan, Zhiwei He, Lingzhong Dong, Yiming Wang, Ruijie Zhao, Tian Xia, Lizhen Xu, Binglin Zhou, Fangqi Li, Zhuosheng Zhang, and 1 others. 2024. R-judge: Benchmarking safety risk awareness for llm agents. arXiv preprint arXiv:2401.10019. · arXiv
  39. Shenglai Zeng, Jiankun Zhang, Pengfei He, Yue Xing, Yiding Liu, Han Xu, Jie Ren, Shuaiqiang Wang, Dawei Yin, Yi Chang, and 1 others. 2024. The good and the bad: Exploring privacy issues in retrieval-augmented generation (rag). arXiv preprint arXiv:2402.16893. · arXiv
  40. Shaoqing Zhang, Zhuosheng Zhang, Kehai Chen, Xin- bei Ma, Muyun Yang, Tiejun Zhao, and Min Zhang. 2024a. Dynamic planning for llm-based graph- ical user interface automation. arXiv preprint arXiv:2410.00467. · arXiv
  41. Yao Zhang, Zijian Ma, Yunpu Ma, Zhen Han, Yu Wu, and Volker Tresp. 2025. Webpilot: A versatile and autonomous multi-agent system for web task execu- tion with strategic exploration. In Proceedings of the AAAI Conference on Artificial Intelligence, vol- ume 39, pages 23378-23386.
  42. Zaibin Zhang, Yongting Zhang, Lijun Li, Hongzhi Gao, Lijun Wang, Huchuan Lu, Feng Zhao, Yu Qiao, and Jing Shao. 2024b. Psysafe: A comprehensive framework for psychological-based attack, defense, and evaluation of multi-agent system safety. arXiv preprint arXiv:2401.11880. · arXiv
  43. Zhexin Zhang, Shiyao Cui, Yida Lu, Jingzhuo Zhou, Junxiao Yang, Hongning Wang, and Minlie Huang. 2024c. Agent-safetybench: Evaluating the safety of llm agents. arXiv preprint arXiv:2412.14470. · arXiv
  44. Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and 1 oth- ers. 2022. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625. · arXiv
  45. Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, and 1 others. 2023. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854. · arXiv
  46. Xuhui Zhou, Hyunwoo Kim, Faeze Brahman, Liwei Jiang, Hao Zhu, Ximing Lu, Frank Xu, Bill Yuchen Lin, Yejin Choi, Niloofar Mireshghallah, and 1 others. 2024. Haicosystem: An ecosystem for sandboxing safety risks in human-ai interactions. arXiv preprint arXiv:2409.16427. · arXiv
  47. Zhenhong Zhou, Zherui Li, Jie Zhang, Yuanhe Zhang, Kun Wang, Yang Liu, and Qing Guo. 2025. Corba: Contagious recursive blocking attacks on multi-agent systems based on large language models. arXiv preprint arXiv:2502.14529. · arXiv
  48. Kunlun Zhu, Hongyi Du, Zhaochen Hong, Xiaocheng Yang, Shuyi Guo, Daisy Zhe Wang, Zhenhailong Wang, Cheng Qian, Robert Tang, Heng Ji, and 1 oth- ers. 2025. Multiagentbench: Evaluating the collabo- ration and competition of llm agents. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8580-8622.
  49. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Univer- sal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043. GPT GPT-5 GPT-5 68.06 ± 2.28 GPT-5 GPT-5-mini 69.44 ± 3.81 GPT-5 GPT-5-nano 72.28 ± 1.80 GPT-5-mini GPT-5-mini 70.19 ± 1.87 GPT-5-mini GPT-5-nano 71.78 ± 4.40 GPT-5-nano GPT-5 44.76 ± 2.40 GPT-5-nano GPT-5-mini 48.68 ± 2.83 GPT-5-nano GPT-5-nano 57.70 ± 2.98 Gemini Gemini-2.5-pro Gemini-2.5-pro 64.22 ± 1.43 Gemini-2.5-pro Gemini-2.5-flash 68.95 ± 2.18 Gemini-2.5-pro Gemini-2.0-flash 48.56 ± 2.95 Gemini-2.5-flash Gemini-2.5-flash 64.37 ± 3.80 Gemini-2.5-flash Gemini-2.0-flash 40.84 ± 4.47 Gemini-2.0-flash Gemini-2.5-pro 29.82 ± 1.47 Gemini-2.0-flash Gemini-2.5-flash 33.81 ± 3.01 Gemini-2.0-flash Gemini-2.0-flash 29.60 ± 3.77 Claude Claude-opus-4.1 Claude-sonnet-4 80.55 ± 1.21 Claude-opus-4.1 Claude-sonnet-3.7 80.03 ± 1.31 Claude-sonnet-4 Claude-sonnet-4 84.90 ± 0.65 Claude-sonnet-4 Claude-sonnet-3.7 79.34 ± 1.56 Claude-sonnet-3.7 Claude-sonnet-3.7 77.42 ± 1.76 Deepseek Deepseek-R1 Deepseek-R1 78.95 ± 2.87 Deepseek-R1 Deepseek-V3 75.00 ± 1.79 Deepseek-V3 Deepseek-R1 78.14 ± 2.86 Deepseek-V3 Deepseek-V3 76.16 ± 2.88 · arXiv

Cited by

See all 1

Metadata

Authors
Dong, S., Zhang, M., He, P., Ma, L., Thuraisingham, B., Liu, H., & Xing, Y.
Venue
Findings of the Association for Computational Linguistics: EACL 2026
Publisher
Association for Computational Linguistics
Published
2026
Pages
4547-4567
DOI
10.18653/v1/2026.findings-eacl.237
Cited by
1
References
8

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: PEAR: Planner-Executor Agent Robustness Benchmark. https://app.cochat.ai/papers/10.18653/v1/2026.findings-eacl.237
The paper
Dong, S., Zhang, M., He, P., Ma, L., Thuraisingham, B., Liu, H., & Xing, Y. (2026). PEAR: Planner-Executor Agent Robustness Benchmark. Findings of the Association for Computational Linguistics: EACL 2026. https://doi.org/10.18653/v1/2026.findings-eacl.237

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.