APRES: An Agentic Paper Revision and Evaluation System
Open accessAbstract
Scientific discoveries must be communicated clearly to realize their full potential. Without effective communication, even the most groundbreaking findings risk being overlooked or misunderstood. The primary way scientists communicate their work and receive feedback from the community is through peer review. However, the current system often provides inconsistent feedback between reviewers, ultimately hindering the improvement of a manuscript and limiting its potential impact. In this paper, we introduce a novel method APRES powered by Large Language Models (LLMs) to update a scientific papers text based on an evaluation rubric. Our automated method discovers a rubric that is highly predictive of future citation counts, and integrate it with APRES in an automated system that revises papers to enhance their quality and impact. Crucially, this objective should be met without altering the core scientific content. We demonstrate the success of APRES, which improves future citation prediction by 19.6% in mean averaged error over the next best baseline, and show that our paper revision process yields papers that are preferred over the originals by human expert evaluators 79% of the time. Our findings provide strong empirical support for using LLMs as a tool to help authors stress-test their manuscripts before submission. Ultimately, our work seeks to augment, not replace, the essential role of human expert reviewers, for it should be humans who discern which discoveries truly matter, guiding science toward advancing knowledge and enriching lives.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The dataset for paper review is drawn from Zhao et al. (2026), which constructs a large-scale benchmark from publicly available submissions and acceptance decisions from recent top-tier machine learning conferences.
“The dataset is drawn from Zhao et al. (2026), which constructs a large-scale benchmark from publicly available submissions and acceptance decisions from recent top-tier machine learning conferences.”
From Hyperagents (2026) · cites this paper as Zhao et al., 2026
The RQGM lets the reviewer, grader, and code reviewer co-evolve with the writers, provers, and coders they score, anchored to APReS decisions, IMO-GradingBench grades, and accept/reject labels from CRAVE.
“The RQGM lets the reviewer, grader, and code reviewer co-evolve with the writers, provers, and coders they score, anchored to APReS decisions [25], IMO-GradingBench grades [7], and accept/reject labels from CRAVE [27].”
From The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators (2026) · cites this paper as [25]
Beyond coding, the DGM-H substantially improves performance on paper review and robotics reward design, with gains transferring to held-out test tasks and significantly outperforming prior self-improving algorithms.
“Beyond coding, the DGM-H substantially improves performance on paper review (Zhao et al., 2026) and robotics reward design (Genesis, 2024), with gains transferring to held-out test tasks and significantly outperforming prior self-improving algorithms, which struggle outside coding unless customized.”
From Hyperagents (2026) · cites this paper as Zhao et al., 2026
The data in the paper review domain are drawn from Zhao et al. (2026).
“The data in this domain are drawn from Zhao et al. (2026).”
From Hyperagents (2026) · cites this paper as Zhao et al. (2026)
Ground-truth labels in the paper review domain correspond to real acceptance decisions from top-tier machine learning conferences, including ICLR 2024/2025 and NeurIPS 2023/2024.
“Ground-truth labels correspond to real acceptance decisions from top-tier machine learning conferences, including ICLR 2024/2025 and NeurIPS 2023/2024.”
From Hyperagents (2026) · cites this paper as Zhao et al. (2026)
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Sorted by citations
The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators
Iacob, A., Jovanović, A., Shen, W. F., Burkhardt, D., Kurmanji, M., Tastan, N., Sani, L., Venanzi, N. A. E., Nie, J., Odonnat, A., et al. (2026) · arXiv · doi:10.48550/arxiv.2606.26294 · Cited by 1 · see citing works
“3.5).”
InfluentialZhang, J., Zhao, B., Yang, W., Foerster, J., Clune, J., Jiang, M., Devlin, S., & Shavrina, T. (2026) · arXiv · doi:10.48550/arxiv.2603.19461
Self-citationPuzzles to Paradigms: A Survey of Human–Agent Collaboration in Scientific Innovation
Liang, Z., Zhang, H., Chen, H., Xiao, T., Ding, J., & Zhang, Y. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.1645
- Authors
- Zhao, B., Zhang, J., Whitehouse, C., Jiang, M., Shvartsman, M., Charnalia, A., Magka, D., Shavrina, T., Dunfield, D., Mac Aodha, O., & Bachrach, Y.
- Affiliations
- University of Edinburgh (GB)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2603.03142
- OpenAlex
- W7133528329
- Cited by
- 0
- References
- 0
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Keywords
- large language models, peer review, citation prediction, assessment rubrics, scientific writing, paper quality improvement, human expert evaluation, science communication
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
