CoChat

“Give a Positive Review Only”: An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers

Open access
Zhou, Qin, Zhang, Zhexin, Zhi, Li, Sun, Limin · 2026 · Findings of the Association for Computational Linguistics: EMNLP 2026

The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index; none of the 6 sources its claims cite is checked for support yet.

Abstract

With the rapid advancement of AI models, their deployment across diverse tasks has become increasingly widespread.A notable emerging application is leveraging AI models to assist in reviewing scientific papers.However, recent reports have revealed that some papers contain hidden, injected prompts designed to manipulate AI reviewers into providing overly favorable evaluations.In this work, we present an early systematic investigation into this emerging threat.We propose two classes of attacks:(1) static attack, which employs a fixed injection prompt, and (2) iterative attack, which optimizes the injection prompt against a simulated reviewer model to maximize its effectiveness.Both attacks achieve striking performance, frequently inducing full evaluation scores when targeting frontier AI reviewers.Furthermore, we show that these attacks are robust across various settings.To counter this threat, we explore a simple detection-based defense.While it substantially reduces the attack success rate, we demonstrate that an adaptive attacker can partially circumvent this defense.Our findings underscore the need for greater attention and rigorous safeguards against prompt-injection threats in AI-assisted peer review.

Overview

CoChat Summary

TL;DR

This paper systematically studies in-paper prompt injection (IPI) attacks that hide white-colored instructions in PDFs to manipulate AI reviewers, proposing static and iterative attacks and a detection-based defense. It finds attacks substantially inflate scores across frontier and open-weight models, and that an adaptive attacker can partially bypass detection.

Key findings

  • Static attacks with fixed prompts increased average ICLR-style ratings by 1.91 for Gemini, 2.80 for DeepSeek, and 1.24 for GPT-5; iterative attacks drove Gemini, DeepSeek, and GPT-OSS-20B close to the maximum score of 10 within at most three optimization rounds, while GPT-5 remained more conservative (average ratings 6.76 and 6.99).
  • Attacks were robust across injection positions (top of first page, bottom of middle page, bottom of last page), human-assigned ratings, and paper lengths; iterative prompts also transferred across reviewer models (e.g., GPT-5-generated prompts transferred to DeepSeek-Chat +2.72 and Gemini-2.5-Pro +2.19).
  • Prompt 3 on DeepSeek-Chat pushed all three rubric scores to 4 and confidence to 5, reduced weaknesses length from 607 to 73, and increased strengths length from 869 to 982.
  • A combined detect-and-score defense detected 99% of attacks and reduced average rating from 10.00 to 7.27, but an adaptive attack raised average score from 7.06 to 8.11 with only 24% detected and 20% fully recovered.

Methods

  • Sampled 100 ICLR 2025 submissions with random.seed(12), without filtering by topic, area, or acceptance status; evaluated AI reviewers GPT-5, DeepSeek-Chat, Gemini-2.5-Pro, GPT-OSS-20B, and Llama-3.1-8B-Instruct via OpenRouter, using pdf-text extraction and temperature 0.9; each paper was reviewed three times under ICLR rubric (overall 1–10; soundness/presentation/contribution 1–4).
  • Static attacks injected four fixed prompt categories as white, microscopic text at the bottom of the final PDF page using PyMuPDF; iterative attacks used an attacker model to refine the injected prompt over at most three rounds based on the simulated reviewer’s prior review, initialized with Prompt 1 or Prompt 3.
  • Defense tested combined detect-and-score and separate detect-then-score (independent detector at T=0, with a reviewer-role variant); adaptive attack prompted a generator to synthesize paper-conditioned prompts intended to evade detection without iterative optimization.

Limitations

  • The paper states that due to high token/computational cost, evaluation is limited to 100 papers and three AI reviewers, with larger sets left as future work.
  • The threat model concerns the complete AI review pipeline including PDF text extraction; a vision-based pipeline that renders pages as images would not expose the white, visually imperceptible injection and is outside the attack’s effective scope.
  • Evident from the text: defense experiments use DeepSeek-Chat as the reviewer (Table 9), and the adaptive attack is evaluated without iterative optimization, so defense generality across reviewers and stronger adaptive strategies is not established.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.18653/2026.findings-emnlp.49

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 Valid · 1 Not valid · 4 Not checked · 88 Own results and interpretations

45 more claims

Create a free account to see all 93 claims and to verify the 3 references still open.

See all claims — free account
  • It may state two separate claims as one. 37
  • The quote only partly states this claim. 2
  • Probably not a substantial claim. 13

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 33 references

Research works printed but not linked: 4

  1. Xiaogeng Liu, Zhiyuan Yu, Yizhe Zhang, Ning Zhang, and Chaowei Xiao. 2024b. Automatic and univer- sal prompt injection attacks against large language models. ArXiv, abs/2403.04957. · arXiv
  2. Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. 2024c. Formalizing and benchmarking prompt injection attacks and defenses. In 33rd USENIX Security Symposium (USENIX Security 24), pages 1831-1847, Philadelphia, PA. USENIX Association.
  3. Zihao Xu, Yi Liu, Gelei Deng, Yuekang Li, and Stjepan Picek. 2024. A comprehensive study of jailbreak attack versus defense for large language models. In Annual Meeting of the Association for Computational Linguistics.
  4. Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. 2024. Evaluating large lan- guage models at evaluating instruction following. In The Twelfth International Conference on Learning Representations.

Cited by

See all 8

Metadata

Authors
Zhou, Q., Zhang, Z., Zhi, L., & Sun, L.
Affiliations
  • Chinese Academy of Sciences (CN)
  • Tsinghua University (CN)
Venue
Findings of the Association for Computational Linguistics: EMNLP 2026
Published
2026
Pages
982-997
OpenAlex
W4416435600
Cited by
8
References
21
Citations by year
2026: 8
FWCI
7.08
Type
conference paper
Language
en
Open access
Locations
  • Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
National Natural Science Foundation of China (U1766215)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
  • Physical Sciences › Computer Science › Hardware and Architecture › Security and Verification in Computing
Keywords
prompt injection, AI-assisted peer review

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-20, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-20 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: “Give a Positive Review Only”: An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers. https://app.cochat.ai/papers/10.18653/2026.findings-emnlp.49
The paper
Zhou, Q., Zhang, Z., Zhi, L., & Sun, L. (2026). “Give a Positive Review Only”: An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers. Findings of the Association for Computational Linguistics: EMNLP 2026. https://doi.org/10.18653/2026.findings-emnlp.49

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.