“Give a Positive Review Only”: An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers
Open accessThe DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index; none of the 6 sources its claims cite is checked for support yet.
Abstract
With the rapid advancement of AI models, their deployment across diverse tasks has become increasingly widespread.A notable emerging application is leveraging AI models to assist in reviewing scientific papers.However, recent reports have revealed that some papers contain hidden, injected prompts designed to manipulate AI reviewers into providing overly favorable evaluations.In this work, we present an early systematic investigation into this emerging threat.We propose two classes of attacks:(1) static attack, which employs a fixed injection prompt, and (2) iterative attack, which optimizes the injection prompt against a simulated reviewer model to maximize its effectiveness.Both attacks achieve striking performance, frequently inducing full evaluation scores when targeting frontier AI reviewers.Furthermore, we show that these attacks are robust across various settings.To counter this threat, we explore a simple detection-based defense.While it substantially reduces the attack success rate, we demonstrate that an adaptive attacker can partially circumvent this defense.Our findings underscore the need for greater attention and rigorous safeguards against prompt-injection threats in AI-assisted peer review.
Overview
CoChat Summary
TL;DR
This paper systematically studies in-paper prompt injection (IPI) attacks that hide white-colored instructions in PDFs to manipulate AI reviewers, proposing static and iterative attacks and a detection-based defense. It finds attacks substantially inflate scores across frontier and open-weight models, and that an adaptive attacker can partially bypass detection.
Key findings
- Static attacks with fixed prompts increased average ICLR-style ratings by 1.91 for Gemini, 2.80 for DeepSeek, and 1.24 for GPT-5; iterative attacks drove Gemini, DeepSeek, and GPT-OSS-20B close to the maximum score of 10 within at most three optimization rounds, while GPT-5 remained more conservative (average ratings 6.76 and 6.99).
- Attacks were robust across injection positions (top of first page, bottom of middle page, bottom of last page), human-assigned ratings, and paper lengths; iterative prompts also transferred across reviewer models (e.g., GPT-5-generated prompts transferred to DeepSeek-Chat +2.72 and Gemini-2.5-Pro +2.19).
- Prompt 3 on DeepSeek-Chat pushed all three rubric scores to 4 and confidence to 5, reduced weaknesses length from 607 to 73, and increased strengths length from 869 to 982.
- A combined detect-and-score defense detected 99% of attacks and reduced average rating from 10.00 to 7.27, but an adaptive attack raised average score from 7.06 to 8.11 with only 24% detected and 20% fully recovered.
Methods
- Sampled 100 ICLR 2025 submissions with random.seed(12), without filtering by topic, area, or acceptance status; evaluated AI reviewers GPT-5, DeepSeek-Chat, Gemini-2.5-Pro, GPT-OSS-20B, and Llama-3.1-8B-Instruct via OpenRouter, using pdf-text extraction and temperature 0.9; each paper was reviewed three times under ICLR rubric (overall 1–10; soundness/presentation/contribution 1–4).
- Static attacks injected four fixed prompt categories as white, microscopic text at the bottom of the final PDF page using PyMuPDF; iterative attacks used an attacker model to refine the injected prompt over at most three rounds based on the simulated reviewer’s prior review, initialized with Prompt 1 or Prompt 3.
- Defense tested combined detect-and-score and separate detect-then-score (independent detector at T=0, with a reviewer-role variant); adaptive attack prompted a generator to synthesize paper-conditioned prompts intended to evade detection without iterative optimization.
Limitations
- The paper states that due to high token/computational cost, evaluation is limited to 100 papers and three AI reviewers, with larger sets left as future work.
- The threat model concerns the complete AI review pipeline including PDF text extraction; a vision-based pipeline that renders pages as images would not expose the white, visually imperceptible injection and is outside the attack’s effective scope.
- Evident from the text: defense experiments use DeepSeek-Chat as the reviewer (Table 9), and the adaptive attack is evaluated without iterative optimization, so defense generality across reviewers and stronger adaptive strategies is not established.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 1 Not valid · 4 Not checked · 88 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Detection-based defenses mitigate static prompts, especially when detection and scoring are decoupled”
§ "Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers > 5 Conclusion
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“although the defense mechanism exhibits high detection accuracy against simple attacks”
§ "Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers > 4.4 Defense and Adaptive Adversarial Attack
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“while one falls below its baseline by more than 1.5 points”
§ "Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers > 4.4 Defense and Adaptive Adversarial Attack
plainstates a causeSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Iterative attacks drive all models except GPT-5 close to the maximum score of 10.”
§ "Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers > 4.2 Main Results
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Detection-based defenses mitigate static prompts”
§ "Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers > 5 Conclusion
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
45 more claims
Create a free account to see all 93 claims and to verify the 3 references still open.
- It may state two separate claims as one. 37
- The quote only partly states this claim. 2
- Probably not a substantial claim. 13
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ "Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers > Questions:
“GPT-5 follows a different pattern and is most sensitive to Prompt 4 (+1.60).”
§ "Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers > 4.3.2 Sensitivity to Decoding Temperature
“Original Paper (No Attack) 7.03 ± 0.02 7.06 ± 0.07 5.73 ± 0.01 6.38 ± 0.04 6.07 ± 0.05 5.12 Static Attack (Prompt 1) 6.92 ± 0.03 7.73 ± 0.05 5.78 ± 0.03 7.58 ± 0.07 8.45 ± 0.07 -Static Attack (Prompt 2) 7.07 ± 0.10 7.99 ± 0.02 6.55 ± 0.06 7.17 ± 0.09 6.96 ± 0.07 -Static Attack (Prompt 3) 8.94 ± 0.16 9.86 ± 0.04 5.93 ± 0.03 7.84 ± 0.09 9.61 ± 0.08 -Static Attack (Prompt 4) 7.71 ± 0.13 8.45 ± 0.07 6.97 ± 0.04 7.19 ± 0.06 9.05 ± 0.13”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ "Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers > 4.3.2 Sensitivity to Decoding Temperature
“Iterative Attack (Prompt 3-based) 9.84 ± 0.03 10.00 ± 0.00 6.99 ± 0.05 8.83 ± 0.05 9.95 ± 0.01”
§ "Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers > 4.3.6 Impact of Attack Iteration Count
“Init: P3 8.94 ± 0.16 9.86 ± 0.04 5.93 ± 0.03 1 9.63 ± 0.01 9.98 ± 0.05 6.37 ± 0.08 2 9.80 ± 0.03 10.00 ± 0.00 6.69 ± 0.02 3 9.84 ± 0.03 10.00 ± 0.00 6.99 ± 0.03”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ "Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers > 4.3.8 Correlation Between Human and AI Reviewer Scores
“DeepSeek (7.03) and Gemini (7.05) score higher on average than humans (5.12), whereas GPT-5 is closer (5.37).”
§ "Give a Positive Review Only": An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers > 4.3.2 Sensitivity to Decoding Temperature
“Original Paper (No Attack) 7.03 ± 0.02 7.06 ± 0.07 5.73 ± 0.01 6.38 ± 0.04 6.07 ± 0.05 5.12”
Citations
Cites
GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak PromptsPreprint
Yu, J., Lin, X., Yu, Z., & Xing, X. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.10253 · Cited by 774
Universal Adversarial Triggers for Attacking and Analyzing NLP
Wallace, E., Feng, S., Kandpal, N., Gardner, M., & Singh, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1221 · Cited by 642
Liang, W., Zhang, Y., Cao, H., Wang, B., Ding, D., Yang, X., Vodrahalli, K., He, S., Smith, D., Yin, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.01783 · Cited by 633
Jailbreaking Black Box Large Language Models in Twenty Queries
Chao, P., Robey, A., Dobriban, E., Hassani, H., Pappas, G. J., & Wong, E. (2023) · arXiv · doi:10.48550/arxiv.2310.08419 · Cited by 522
Automatic and Universal Prompt Injection Attacks against Large Language ModelsPreprint
Liu, X., Yu, Z., Zhang, Y., Zhang, N., & Xiao, C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.04957 · Cited by 285
AgentReview: Exploring Peer Review Dynamics with LLM Agents
Jin, Y., Zhao, Q., Wang, Y., Chen, H., Zhu, K., Xiao, Y., & Wang, J. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.70 · Cited by 218
Universal and Transferable Adversarial Attacks on Aligned Language Models
Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z., & Fredrikson, M. (2023) · arXiv · doi:10.48550/arxiv.2307.15043 · Cited by 197
Jailbreaking Black Box Large Language Models in Twenty Queries
Chao, P., Robey, A., Dobriban, E., Hassani, H., Pappas, G. J., & Wong, E. (2025) · 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml64287.2025.00010 · Cited by 193
LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks
Bavaresco, A., Bernardi, R., Bertolazzi, L., Elliott, D., Fernández, R., Gatt, A., Ghaleb, E., Giulianelli, M., Hanna, M., Koller, A., et al. (2024) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-c-000794254 · Cited by 181
Can LLM feedback enhance review quality? A randomized study of 20K reviews at ICLR 2025
Thakkar, N., Yuksekgonul, M., Silberg, J., Garg, A., Peng, N., Sha, F., Yu, R., Vondrick, C., & Zou, J. (2025) · arXiv · doi:10.48550/arxiv.2504.09737 · Cited by 149
Research works printed but not linked: 4
- Xiaogeng Liu, Zhiyuan Yu, Yizhe Zhang, Ning Zhang, and Chaowei Xiao. 2024b. Automatic and univer- sal prompt injection attacks against large language models. ArXiv, abs/2403.04957. · arXiv
- Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. 2024c. Formalizing and benchmarking prompt injection attacks and defenses. In 33rd USENIX Security Symposium (USENIX Security 24), pages 1831-1847, Philadelphia, PA. USENIX Association.
- Zihao Xu, Yi Liu, Gelei Deng, Yuekang Li, and Stjepan Picek. 2024. A comprehensive study of jailbreak attack versus defense for large language models. In Annual Meeting of the Association for Computational Linguistics.
- Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. 2024. Evaluating large lan- guage models at evaluating instruction following. In The Twelfth International Conference on Learning Representations.
Cited by
No Hidden Prompts Needed! You Can Game AI Peer Review with Presentation-Only RevisionsPreprint
Yang, X., Sha, Z., Li, J., Yu, J., Sun, Y., Zhao, M., Fang, J., Guo, X., Wu, Y., Hu, X., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.13044 · Cited by 9
When and how to disclose AI use in academic peer review
Masters, K., & Cleland, J. (2026) · Medical Teacher · doi:10.1080/0142159x.2025.2609741 · Cited by 3
The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly PublishingPreprint
Wang, C., Li, M., Braimah, A., Fan, C., Wang, T., Guan, W., Zhang, R., Zhou, T., & Zhou, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.07713 · Cited by 3
Flooded GatewaysPreprint
Knight, S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6297213 · Cited by 1
Georgantas, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.15887 · Cited by 1
Metadata
- Authors
- Zhou, Q., Zhang, Z., Zhi, L., & Sun, L.
- Affiliations
- Chinese Academy of Sciences (CN)
- Tsinghua University (CN)
- ORCID
- Sun, Limin
- Venue
- Findings of the Association for Computational Linguistics: EMNLP 2026
- Published
- 2026
- Pages
- 982-997
- OpenAlex
- W4416435600
- Cited by
- 8
- References
- 21
- Citations by year
- 2026: 8
- FWCI
- 7.08
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Natural Science Foundation of China (U1766215)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Physical Sciences › Computer Science › Hardware and Architecture › Security and Verification in Computing
- Keywords
- prompt injection, AI-assisted peer review
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-20, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-20 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: “Give a Positive Review Only”: An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers. https://app.cochat.ai/papers/10.18653/2026.findings-emnlp.49
- The paper
- Zhou, Q., Zhang, Z., Zhi, L., & Sun, L. (2026). “Give a Positive Review Only”: An Early Investigation Into In-Paper Prompt Injection Attacks and Defenses for AI Reviewers. Findings of the Association for Computational Linguistics: EMNLP 2026. https://doi.org/10.18653/2026.findings-emnlp.49
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.