Exploiting large language models in peer review: indirect prompt injection attacks and integrity probes
Open accessPublished version of a preprint
The DOI matches its Crossref record; 97 of the 98 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Large language models are beginning to enter peer review as tools for summarizing manuscripts, drafting evaluations, and reducing reviewer workload. Yet this use creates a security problem specific to evaluative settings: the manuscript being judged can also contain hidden instructions that shape the model’s judgment. We investigate this risk through indirect prompt injection, where hidden text embedded in a manuscript is processed like a benign prompt by a public chatbot during review generation. We introduce an Author-Reviewer-Organizer framework that distinguishes author-side manipulation from organizer-side integrity checks, and use it to evaluate both offensive payloads, which steer reviews toward positive or negative judgments, and integrity-probing payloads, which trigger refusal, watermarking, or external-site redirection. The redirection payload is introduced here as a novel integrity-check mechanism. In experiments on 100 OpenReview papers, two public chatbot systems, five payload families, two custom prompt approaches, three injection positions, and five repeated runs per condition, we generate 42000 model outputs. Hidden instructions are highly effective: positive steering, refusal, and external-site redirection exceed 98% success on both systems; watermarking remains high but less reliable, reaching 94.27% on ChatGPT and 88.17% on Gemini; and negative steering is near-ceiling on ChatGPT but lower on Gemini. Vulnerability varies with payload design, model, and document position. These findings show that current models exhibit “contextual blindness”: they do not reliably separate manuscript content from embedded control text. This weakness threatens the validity of AI-assisted review, while also creating a dual-use channel that organizers may repurpose for detection. Peer-review systems that rely on public LLMs therefore require stronger safeguards between document content and instructions before they can be trusted in evaluative workflows.
Checked 2026-11-20 · 1 reference not checked yet
Overview
CoChat Summary
TL;DR
This paper studies indirect prompt injection in AI-assisted peer review by embedding hidden instructions in manuscript PDFs and asking public ChatGPT and Gemini chatbots to review them. Across 42,000 outputs, hidden instructions frequently steered reviews, forced refusals, inserted watermarks, or redirected users, revealing what the authors call “contextual blindness” in current LLM pipelines.
Key findings
- Both systems were highly vulnerable: pooled overall attack success rate was 98.34% (20,651/21,000) for ChatGPT and 94.02% (19,744/21,000) for Gemini. Positive steering, refusal, and external-site redirection exceeded 98% success on both systems; watermarking reached 94.27% on ChatGPT and 88.17% on Gemini; negative steering was 98.97% on ChatGPT but 86.87% on Gemini.
- Vulnerability varied by payload, model, and injection locus. ChatGPT pooled ASR was 99.87% at L1, 95.19% at L2, and 99.96% at L1+L2; Gemini was 90.89%, 95.74%, and 95.43% across the same loci. Narrative watermarking dropped to 72.8% at L2 on ChatGPT; policy-puppetry negative steering fell to 50.0% at L1 on Gemini.
- Leetspeak-obfuscated Refusal and Watermark payloads achieved 100% ASR on all reviewing prompts, matching plaintext performance. In a shared-benchmark comparison, the authors’ Refusal and Positive Steering payloads reached 100% ASR on both models, versus 44.6%/51.7% for Collu et al.’s Ignore and 64.2–68.3%/50.4–67.5% for Exploit-1/2.
- Steering shifted review polarity: positive steering reached VADER mean 0.991 ± 0.001 on both models; negative steering reached -0.769 ± 0.008 on ChatGPT and -0.384 ± 0.022 on Gemini, against human reference means of 0.827 (accepted) and 0.816 (rejected). Successful outputs almost always reproduced the injected opening (99.93% ChatGPT positive, 100.00% ChatGPT negative, 99.51% Gemini positive, 97.51% Gemini negative).
Methods
- Design: Author-Reviewer-Organizer (ARO) framework; five payload families (positive steering, negative steering, refusal, watermarking, external-site redirection), two prompt approaches (Narrative Prompting, Policy Puppetry), three injection positions (L1 first page, L2 last page, L1+L2), five repeated runs per condition; hidden text embedded via PDF render-mode 3 Tr overlay using ReportLab/pypdf.
- Data/sample: OpenReview corpus restricted to papers published no later than 30 Nov 2022; quality filter retained reviews above 75th percentile length (avg >3279 characters), yielding 4,900 papers and >17,000 reviews; final 100 manuscripts (50 accepted, 50 rejected) selected deterministically by total review length within decision class, spanning ICLR 2019–2022 and NeurIPS 2021, with 420 individual reviews (avg 6,003 characters).
- Procedure/analysis: automated Selenium upload-and-query workflow on public ChatGPT (GPT-5.3 Instant) and Gemini (Gemini-3-flash-preview) web interfaces, logged-out/private, no custom configuration; 42,000 outputs total. Watermark and external-site redirection scored by exact checks; refusal by primary LLM judge (Qwen3.5-27B); steering by consensus of two LLM judges (Qwen3.5-27B and Gemma-4-31B-IT), with VADER and a DistilBERT academic sentiment classifier as robustness checks; pooled success rates and Wilson intervals reported.
Limitations
- Paper-stated: two public chatbots in a computer science peer-review setting, so rates are not fixed properties of other disciplines, proprietary systems, or future model versions; only three reviewer request templates, so attack performance may shift under other prompt formulations; carrier restricted to invisible PDF text via rendering-mode manipulation, so other carriers may produce different vulnerability profiles; the 100-paper sample is not representative of OpenReview submissions, weighted toward unusually detailed reviews and concentrated in ICLR 2019–2022 and NeurIPS 2021.
- Paper-stated: the deceptive external-site redirection variant was not empirically evaluated; whether deceptive framing increases link-following remains untested and would require a user study. Sampling parameters (temperature, top-p, max tokens) are not exposed through the public web interfaces, so only default web-interface behavior is reported.
- Evident from text: the sample selection by review length introduces selection bias (the paper flags this), and all NeurIPS entries in the sample are accepted papers because no rejected NeurIPS 2021 submissions entered the top-50 rejected set; the study measures model outputs, not actual reviewer behavior, and the external-site redirection signal depends on a human reviewer interacting with the model’s output.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures







Verification
Claims the paper makes
No claims could be anchored to a verbatim quote in the text.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Exploiting large language models in peer review: indirect prompt injection attacks and integrity probes > Stealthiness results
“Across both attack types and both models, leet-encoded payloads achieved 100% ASR on all reviewing prompts, matching the performance of their plaintext counterparts evaluated using the same reviewing prompts (R0-R3) adopted from Collu et al. (2026) and reported in Section Baseline comparison.”
§ Exploiting large language models in peer review: indirect prompt injection attacks and integrity probes > Direct comparison on shared benchmark
“Ours: Watermark (Narrative) 99.6±0.6 78.4±2.9Ours: Watermark (Policy P.) 99.8±0.5 99.4±0.6”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Exploiting large language models in peer review: indirect prompt injection attacks and integrity probes > Taxonomy of payloads
“The External Site Redirection payload extends the family of organizer-side integrity checks with a detection channel that operates outside the generated review.”
§ Exploiting large language models in peer review: indirect prompt injection attacks and integrity probes > 2
“| External-site redirection | Offensive | standard only | 500 | 3000 |”
Citations
Cites
VADER: a parsimonious rule-based model for sentiment analysis of social media text
Hutto, C., & Gilbert, E. (2014) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v8i1.14550 · Cited by 6,087
Bibliometrics: The Leiden Manifesto for Research Metrics
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015) · Nature · doi:10.1038/520429a · Cited by 2,828
Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22784 · Cited by 1,000
Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023) · Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3605764.3623985 · Cited by 667
Benchmarking and Defending against Indirect Prompt Injection Attacks on Large Language Models
Yi, J., Xie, Y., Zhu, B., Kıcıman, E., Sun, G., Xie, X., & Wu, F. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3690624.3709179 · Cited by 632
Chance and Consensus in Peer Review
Cole, S., Cole, J. R., & Simon, G. A. (1981) · Science · doi:10.1126/science.7302566 · Cited by 623
Bornmann, L. (2011) · Annual Review of Information Science and Technology · doi:10.1002/aris.2011.1440450112 · Cited by 567
Perplexity—a measure of the difficulty of speech recognition tasks
Jelinek, F., Mercer, R. L., Bahl, L. R., & Baker, J. K. (1977) · The Journal of the Acoustical Society of America · doi:10.1121/1.2016299 · Cited by 398
Does Prompt Formatting Have Any Impact on LLM Performance?Preprint
He, J., Rungta, M., Koleczek, D., Sekhon, A., Wang, F. X., & Hasan, S. A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.10541 · Cited by 339
Duration and quality of the peer review process: the author’s perspective
Huisman, J., & Smits, J. (2017) · Scientometrics · doi:10.1007/s11192-017-2310-5 · Cited by 303
Research works printed but not linked: 3
- Liu, Y., Jia, Y., Geng, R., Jia, J., & Gong, N.Z. (2024). Formalizing and benchmarking prompt injection attacks and defenses. 33rd USENIX Security Symposium (USENIX Security 24) (pp. 1831–1847). Philadelphia, PAUSENIX Association. https://www.usenix.org/conference/usenixsecurity24/presentation/liu-yupei
- Vaswani, A., Shazeer, N. M., Parmar, N., Uszkoreit, J., Jones, L., & Gomez, A. N. (2017). Polosukhin, I. Attention is all you need. NIPS, 30,
- Wu, X., Wang, Y., Jegelka, S., & Jadbabaie, A. (2025). On the emergence of position bias in transformers. A. Singh et al. (Eds.), Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025 (Vol. 267). PMLR / OpenReview.net. https://proceedings.mlr.press/v267/wu25ad.html
Cited by
The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly PublishingPreprint
Wang, C., Li, M., Braimah, A., Fan, C., Wang, T., Guan, W., Zhang, R., Zhou, T., & Zhou, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.07713 · Cited by 3
The Arbiter Agent: Continually Monitoring Multi-Agent Conversations to Detect Emergent MisalignmentPreprint
Tonini, F., Torrielli, F., Lautrup, A. D., Schneider–Kamp, P., Çelikok, M. M., & Poech, L. G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2606.10747 · Cited by 1
Self-citationAuthors, reviewers, editors… smitten and bitten by the artificial intelligence (AI) bug!
Kurdi, M. S., Gangakhedkar, G. R., & Sasidharan, S. (2026) · Indian Journal of Anaesthesia · doi:10.4103/ija.ija_1374_26
Selecting The Most Informative Tokens in Natural Language AutoencodersPreprint
Torrielli, F., Barmina, G., Núñez, A. B., Rapp, A., Di, L., Schneider–Kamp, P., & Poech, L. G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.37040
Self-citationChowdhury, A., Joti, T. J., Afikuzzaman, M., Bithi, A. N., Hafiz, M. F. B., & Khan, N. A. (2026) · International Journal of Information Security · doi:10.1007/s10207-026-01339-6
Metadata
- Authors
- Torrielli, F., Locci, S., Rapp, A., & Di Caro, L.
- Affiliations
- University of Turin (IT)
- Corresponding author
- Torrielli, Federico
- Locci, Stefano
- Rapp, Amon
- Venue
- Scientometrics
- Published
- 2026-07
- Volume
- 131
- Issue
- 7
- Pages
- 4889-4951
- ISSN
- 0138-9130 (ISSN-L)
- 1588-2861
- OpenAlex
- W7168048551
- Cited by
- 8
- References
- 93
- Citations by year
- 2026: 8
- FWCI
- 7.74
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor, 10 d)
- Locations
- Scientometrics (publishedVersion, cc-by)
- Research Square (submittedVersion, cc-by)
- Institutional Research Information System University of Turin (University of Turin) (submittedVersion, cc-by)
- Funders
- NextGenerationEU
- Università degli Studi di Torino
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Keywords
- large language models, peer review, prompt injection, indirect prompt injection, AI-assisted review, watermarking, ChatGPT, Gemini
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-20 (model deepseek-v4.1-flash, prompt extract-v3).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Exploiting large language models in peer review: indirect prompt injection attacks and integrity probes. https://app.cochat.ai/papers/10.1007/s11192-026-05695-x
- The paper
- Torrielli, F., Locci, S., Rapp, A., & Di Caro, L. (2026). Exploiting large language models in peer review: indirect prompt injection attacks and integrity probes. Scientometrics. https://doi.org/10.1007/s11192-026-05695-x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.