Unveiling the Achilles' Heel of NLG Evaluators: A Unified Adversarial Framework Driven by Large Language Models
Open accessAbstract
The automatic evaluation of natural language generation (NLG) systems presents a long-lasting challenge. Recent studies have highlighted various neural metrics that align well with human evaluations. Yet, the robustness of these evaluators against adversarial perturbations remains largely under-explored due to the unique challenges in obtaining adversarial data for different NLG evaluation tasks. To address the problem, we introduce AdvEval, a novel black-box adversarial framework against NLG evaluators. AdvEval is specially tailored to generate data that yield strong disagreements between human and victim evaluators. Specifically, inspired by the recent success of large language models (LLMs) in text generation and evaluation, we adopt strong LLMs as both the data generator and gold evaluator. Adversarial data are automatically optimized with feedback from the gold and victim evaluator. We conduct experiments on 12 victim evaluators and 11 NLG datasets, spanning tasks including dialogue, summarization, and question evaluation. The results show that AdvEval can lead to significant performance degradation of various victim metrics, thereby validating its efficacy.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLM-as-a-Judge systems are susceptible to adversarial attacks that can manipulate evaluation outcomes through subtle perturbations.
“However, these systems are susceptible to adversarial attacks that can manipulate evaluation outcomes, raising critical concerns about their robustness and trustworthiness.”
From LLMs Cannot Reliably Judge (Yet?): A Comprehensive Assessment on the Robustness of LLM-as-a-Judge (2025) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 12
- Jacopo Amidei, Paul Piwek, and Alistair Willis. 2018. Rethinking the agreement in human evaluation tasks. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3318-3329, Santa Fe, New Mexico, USA. Association for Com- putational Linguistics.
- Yonatan Belinkov and Yonatan Bisk. 2018. Synthetic and natural noise both break neural machine transla- 1367 tion. In International Conference on Learning Rep- resentations.
- Javid Ebrahimi, Daniel Lowd, and Dejing Dou. 2018a. On adversarial examples for character-level neural machine translation. In Proceedings of the 27th Inter- national Conference on Computational Linguistics, pages 653-663, Santa Fe, New Mexico, USA. Asso- ciation for Computational Linguistics.
- Xiaoxue Gao, Xiaohai Tian, Rohan Kumar Das, Yi Zhou, and Haizhou Li. 2019. Speaker- independent spectral mapping for speech-to-singing conversion. In 2019 Asia-Pacific Signal and Infor- mation Processing Association Annual Summit and Conference (APSIPA ASC), pages 159-164. · DOI
- Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. 2019a. Textbugger: Generating adversarial text against real-world applications. In 26th Annual Network and Distributed System Security Symposium, NDSS 2019, San Diego, California, USA, February 24-27, 2019. The Internet Society.
- Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. 2019b. Textbugger: Generating adversarial text against real-world applications. In Annual Net- work and Distributed System Security Symposium. The Internet Society.
- Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. DailyDialog: A manually labelled multi-turn dialogue dataset. In Proceedings of the Eighth International Joint Conference on Nat- ural Language Processing (Volume 1: Long Papers), pages 986-995, Taipei, Taiwan. Asian Federation of Natural Language Processing.
- Chin-Yew Lin. 2004. ROUGE: A package for auto- matic evaluation of summaries. In Text Summariza- tion Branches Out, pages 74-81, Barcelona, Spain. Association for Computational Linguistics.
- Shaolin Su, Qingsen Yan, Yu Zhu, Cheng Zhang, Xin Ge, Jinqiu Sun, and Yanning Zhang. 2020. Blindly assess image quality in the wild guided by a self- adaptive hyper network. In Proceedings of the IEEE/CVF conference on computer vision and pat- tern recognition, pages 3667-3676.
- Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021. BARTScore: Evaluating generated text as text gener- ation. In Advances in Neural Information Processing Systems.
- Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Eval- uating text generation with bert. In International Conference on Learning Representations.
- Xiaoqing Zheng, Jiehang Zeng, Yi Zhou, Cho-Jui Hsieh, Minhao Cheng, and Xuanjing Huang. 2020. Evaluat- ing and enhancing the robustness of neural network- based dependency parsing models with adversarial examples. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 6600-6610, Online. Association for Computa- tional Linguistics. · DOI
Cited by
Sorted by citations
Finding Blind Spots in Evaluator LLMs with Interpretable Checklists
Doddapaneni, S., Khan, M. S. U. R., Verma, S., & Khapra, M. M. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.911 · Cited by 9 · see citing works
LLMs Cannot Reliably Judge (Yet?): A Comprehensive Assessment on the Robustness of LLM-as-a-Judge
Li, S., Xu, C., Wang, J., Gong, X., Chen, C., Zhang, J., Wang, J., Lam, K.-Y., & Ji, S. (2025) · arXiv · doi:10.48550/arxiv.2506.09443
“Heuristic-based adversaries, lacking direct access to the evaluation metric m , craft adversarial inputs using indirect cues, such as score feedback [10] or prompt manipulation techniques [14], [30]–[32], [39], [40].”
Influential
- Authors
- Chen, Y., Zhang, C., Luo, D., D'Haro, L. F., Tan, R. T., & Li, H.
- Affiliations
- National University of Singapore (SG)
- Universidad Politécnica de Madrid (ES)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- Pages
- 1359-1375
- arXiv
- 2405.14646
- OpenAlex
- W4402684273
- Cited by
- 0
- References
- 0
- FWCI
- 0.00
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- dx.doi.org
- Licence
- Locations
- Findings of the Association for Computational Linguistics: ACL (Findings of ACL) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Comunidad de Madrid
- European Commission (101071191)
- Universidad Politécnica de Madrid
- European Regional Development Fund (MCIN/AEI/10)
- Agencia Estatal de Investigación (13039, 10.13039, AEI/10)
- Shenzhen Science and Technology Innovation Program
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- natural language generation evaluation, adversarial evaluation, black-box attacks, large language models, dialogue generation, summarization, question evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
