CoChat
Sign Up
PreprintCited by 1Artificial Intelligence ApplicationsEthics and Social Impacts of AIAcademic Writing and Publishing

AI for Auto-Research: Roadmap & User Guide

Open access
Kong, Lingdong, Sun, Xian, Chow, Wei, Li, Linfeng, Lin, Kevin Qinghong, Zhang, Xuan Billy, Wang, Song, Li, Rong +12 more·2026·arXiv
DOI 10.48550/arxiv.2605.18661arXiv 2605.18661
Open this paper in CoChat — free accountOpen PDF

Abstract

AI-assisted research is crossing a threshold: fully automated systems can now generate research papers for as little as $15, while long-horizon agents can execute experiments, draft manuscripts, and simulate critique with minimal human input. Yet this productivity frontier exposes a deeper integrity problem: under scientific pressure, even frontier LLMs still fabricate results, miss hidden errors, and fail to judge novelty reliably. Studying developments through April 2026, we present an end-to-end analysis of AI across the complete research lifecycle, organized into four epistemological phases: Creation (idea generation, literature review, coding & experiments, tables & figures), Writing (paper writing), Validation (peer review, rebuttal & revision), and Dissemination (posters, slides, videos, social media, project pages, and interactive agents). We identify a sharp, stage-dependent boundary between reliable assistance and unreliable autonomy: AI excels at structured, retrieval-grounded, and tool-mediated tasks, but remains fragile for genuinely novel ideas, research-level experiments, and scientific judgment. Generated ideas often degrade after implementation, research code lags far behind pattern-matching benchmarks, and end-to-end autonomous systems have not yet consistently reached major-venue acceptance standards. We further show that greater automation can obscure rather than eliminate failure modes, making human-governed collaboration the most credible deployment paradigm. Finally, we provide a structured taxonomy, benchmark suite, and tool inventory, cross-stage design principles, and a practitioner-oriented playbook, with resources maintained at our project page.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2605.18661

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Providing identical context to all agents wastes tokens and can introduce role-irrelevant information, which may degrade reasoning quality.

    “Providing identical context to all agents wastes tokens (Kong et al., 2026; Tan et al., 2025a; Li et al., 2025a) and can introduce role-irrelevant information (Ji et al., 2025; Niu et al., 2025a; Liu et al., 2025c), which may degrade reasoning quality (Yang et al., 2025a; Liu et al., 2024a; Zhang et al., 2026; Zhao et al., 2025; Tan et al., 2025c).”

    From RCR-Router: Efficient Role-Aware Context Routing for Multi-Agent LLM Systems with Structured Memory (2025) · cites this paper as Kong et al., 2026

    Kong et al. is cited for the first clause: that providing identical context to all agents wastes tokens. Section 7.1.4 (p. 34) discusses duplicated work and verbose deliberation in multi-agent systems, but it does not address identical context or its token cost. Searches for shared context and token overhead found no passage establishing that point; the copy is truncated. (LLM, conf=0.72)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 312

  1. [2][2] S. Afroz et al. 'AI slop is DDoSing open source': Understanding the impact of AI-generated contributions on open source sustainability. arXiv preprint arXiv:2607.04003 , 2026. URL https://arxiv.org/abs/2607.04003 . - · arXiv
  2. [3][3] S. Agarwal, G. Sahu, A. Puri, I. H. Laradji, K. D. Dvijotham, J. Stanley, L. Charlin, and C. Pal. LitLLM: A toolkit for literature review with large language models. arXiv preprint arXiv:2402.01788 , 2024. URL https://arxiv.org/abs/2402.01788 . - · arXiv
  3. [4][4] T. Aggarwal and A. Bhand. PASS: Presentation automation for slide generation and speech. arXiv preprint arXiv:2501.06497 , 2025. URL https://arxiv.org/abs/2501.06497 . - · arXiv
  4. [5][5] A. Ajith, M. Xia, A. Chevalier, T. Goyal, D. Chen, and T. Gao. LitSearch: A retrieval benchmark for scientific literature search. In Conference on Empirical Methods in Natural Language Processing , 2024. URL https://arxiv.org/abs/2407.18940 . - · arXiv
  5. [6][6] I. Al Azher, M. J. Mokarrama, Z. Guo, S. R. Choudhury, and H. Alhoori. FutureGen: A RAG-based approach to generate the future work of scientific article. arXiv preprint arXiv:2503.16561 , 2025. URL https://arxiv.org/ abs/2503.16561 . - · arXiv
  6. [7][7] I. Al Azher, Z. Guo, and H. Alhoori. Multi-agent LLMs for generating research limitations. arXiv preprint arXiv:2601.11578 , 2026. URL https://arxiv.org/abs/2601.11578 . - · arXiv
  7. [8][8] Alibaba NLP. Tongyi DeepResearch: An agentic LLM for long-horizon deep information seeking. https: //github.com/Alibaba-NLP/DeepResearch , 2025. URL https://arxiv.org/abs/2510.24701 . - · arXiv
  8. [10][10] A. Asai, J. He, R. Shao, W. Shi, A. Singh, J. C. Chang, K. Lo, L. Soldaini, S. Feldman, M. D'Arcy, D. Wadden, M. Latzke, J. Sparks, J. D. Hwang, V. Kishore, M. Tian, P. Ji, S. Liu, H. Tong, B. Wu, Y. Xiong, L. Zettlemoyer, G. Neubig, D. S. Weld, D. Downey, W. tau Yih, P. W. Koh, and H. Hajishirzi. Synthesizing scientific literature with retrieval-augmented language models. Nature , 650:857-863, 2026. URL https://arxiv.org/abs/2411.14199 . - · arXiv
  9. [11][11] J. Baek, S. K. Jauhar, S. Cucerzan, and S. J. Hwang. ResearchAgent: Iterative research idea generation over scientific literature with large language models. In Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 6709-6738, 2025. URL https://arxiv.org/abs/2404.07738 . - · arXiv
  10. [12][12] C. Beger and C.-L. Henneking. Citegeist: Automated generation of related work analysis on the arXiv corpus. arXiv preprint arXiv:2503.23229 , 2025. URL https://arxiv.org/abs/2503.23229 . - · arXiv
  11. [13][13] J. Belouadi, A. Lauscher, and S. Eger. AutomaTikZ: Text-guided synthesis of scientific vector graphics with TikZ. In International Conference on Learning Representations , 2024. URL https://arxiv.org/abs/2310.00367 . - · arXiv
  12. [14][14] J. Belouadi, S. P. Ponzetto, and S. Eger. DeTikZify: Synthesizing graphics programs for scientific figures and sketches with TikZ. In Advances in Neural Information Processing Systems , volume 37, pages 85074-85108, 2024. URL https://arxiv.org/abs/2405.15306 . - · arXiv
  13. [15][15] H. Bisht, V. Kumar, K. M. Jablonka, Mausam, and N. M. A. Krishnan. Agentic AI scientists are not built for autonomous scientific discovery. arXiv preprint arXiv:2605.08956 , 2026. URL https://arxiv.org/abs/2605.08956 . - · arXiv
  14. [16][16] J. Biswas et al. AI-assisted peer review at scale: The AAAI-26 AI review pilot. arXiv preprint arXiv:2604.13940 , 2026. URL https://arxiv.org/abs/2604.13940 . - · arXiv
  15. [18][18] D. A. Boiko, R. MacKnight, B. Kline, and G. Gomes. Autonomous chemical research with large language models. Nature , 624(7992):570-578, 2023. URL https://arxiv.org/abs/2304.05332 . - · arXiv
  16. [19][19] J. Bragg, M. D'Arcy, N. Balepur, D. Bareket, B. Dalvi, S. Feldman, D. Haddad, J. D. Hwang, P. Jansen, V. Kishore, B. P. Majumder, A. Naik, S. Rahamimov, K. Richardson, A. Singh, H. Surana, A. Tiktinsky, R. Vasu, G. Wiener, C. Anastasiades, S. Candra, J. Dunkelberger, D. Emery, R. Evans, M. Hamada, R. Huff, R. Kinney, M. Latzke, J. Lochner, R. Lozano-Aguilera, C. Nguyen, S. Rao, A. Tanaka, B. Vlahos, P. Clark, D. Downey, Y. Goldberg, A. Sabharwal, and D. S. Weld. AstaBench: Rigorous benchmarking of AI agents with a scientific research suite. In International Conference on Learning Representations , 2026. URL https://arxiv.org/abs/2510.21652 . - · arXiv
  17. [20][20] A. M. Bran, S. Cox, O. Schilter, C. Baldassari, A. D. White, and P. Schwaller. ChemCrow: Augmenting large language models with chemistry tools. Nature Machine Intelligence , 6(5):525-535, 2024. URL https: //arxiv.org/abs/2304.05376 . -
  18. [22][22] J. Chai, S. Tang, R. Ye, Y. Du, X. Zhu, M. Zhou, Y. Wang, W. E, Y. Zhang, L. Zhang, and S. Chen. SciMaster: Towards general-purpose scientific AI agents, part I. X-Master as foundation: Can we lead on humanity's last exam? arXiv preprint arXiv:2507.05241 , 2025. URL https://arxiv.org/abs/2507.05241 . - · arXiv
  19. [23][23] J. S. Chan, N. Chowdhury, O. Jaffe, J. Aung, D. Sherburn, E. Mays, G. Starace, K. Liu, L. Maksin, T. Patwardhan, L. Weng, and A. Madry. MLE-Bench: Evaluating machine learning agents on machine learning engineering. In International Conference on Learning Representations , 2025. URL https://arxiv.org/abs/2410.07095 . - · arXiv
  20. [24][24] C.-C. Chen and I. Gurevych. Commitment checklist: Auditing author commitments in peer review. arXiv preprint arXiv:2603.00003 , 2026. URL https://arxiv.org/abs/2603.00003 . - · arXiv
  21. [25][25] D. Chen. AI-generated figures in academic publishing: Policies, tools, and practical guidelines. arXiv preprint arXiv:2603.16159 , 2026. URL https://arxiv.org/abs/2603.16159 . - · arXiv
  22. [26][26] D. Chen et al. Can AI draw science? a benchmark for evaluating scientific figure generation by text-to-image and multimodal models. arXiv preprint arXiv:2606.28406 , 2026. URL https://arxiv.org/abs/2606.28406 . - · arXiv
  23. [27][27] G. Chen, J. Chen, L. Chen, J. Zhao, F. Meng, W. X. Zhao, R. Song, C. Chen, J.-R. Wen, and K. Jia. Toward autonomous long-horizon engineering for ML research. arXiv preprint arXiv:2604.13018 , 2026. URL https://arxiv.org/abs/2604.13018 . - · arXiv
  24. [28][28] G. Chen, F. Meng, J. Zhao, M. Li, D. Cheng, H. Song, J. Chen, Y. Lin, H. Chen, X. Zhao, R. Song, C. Liu, C. Chen, K. Jia, and J.-R. Wen. BeyondSWE: Can current code agent survive beyond single-repo bug fixing? arXiv preprint arXiv:2603.03194 , 2026. URL https://arxiv.org/abs/2603.03194 . - · arXiv
  25. [29][29] H. Chen, M. Xiong, Y. Lu, W. Han, A. Deng, Y. He, J. Wu, Y. Li, Y. Liu, and B. Hooi. MLR-Bench: Evaluating AI agents on open-ended machine learning research. arXiv preprint arXiv:2505.19955 , 2025. URL https://arxiv.org/abs/2505.19955 . - · arXiv
  26. [30][30] H. Chen, X. Zhou, W. Zhou, X. Shao, Y. Zhu, B. Wang, J. Hong, A. Jia, and F. Wu. X+Slides: Benchmarking audience-conditioned slide generation. arXiv preprint arXiv:2606.19256 , 2026. URL https://arxiv.org/abs/2606. 19256 . - · arXiv
  27. [31][31] N. Chen, A. H. Lin, J. Wu, J. Hou, Z. Zhang, Q. Wang, X. Wang, and B. He. XtraGPT: Context-aware and controllable academic paper revision. arXiv preprint arXiv:2505.11336 , 2025. URL https://arxiv.org/abs/2505. 11336 . - · arXiv
  28. [32][32] Q. Chen, M. Yang, L. Qin, J. Liu, Z. Yan, J. Guan, D. Peng, Y. Ji, H. Li, M. Hu, Y. Zhang, Y. Liang, Y. Zhou, J. Wang, Z. Chen, and W. Che. AI4Research: A survey of artificial intelligence for scientific research. arXiv preprint arXiv:2507.01903 , 2025. URL https://arxiv.org/abs/2507.01903 . - · arXiv
  29. [33][33] S. Chen, J. Lai, J. Gao, H. Shi, Z. Liu, T. Ye, J. Luo, X. Wei, and L. Zhu. PosterOmni: Generalized artistic poster creation via task distillation and unified reward feedback. arXiv preprint arXiv:2602.12127 , 2026. URL https://arxiv.org/abs/2602.12127 . - · arXiv
  30. [34][34] S. Chen, J. Lai, J. Gao, T. Ye, H. Chen, H. Shi, S. Shao, Y. Lin, S. Fei, Z. Xing, Y. Jin, J. Luo, X. Wei, and L. Zhu. PosterCraft: Rethinking high-quality aesthetic poster generation in a unified framework. In International Conference on Learning Representations , 2026. URL https://arxiv.org/abs/2506.10741 . - · arXiv
  31. [35][35] S. Chen, S. Zhong, D. P. Brumby, and A. L. Cox. What happens when reviewers receive AI feedback in their reviews? In CHI Conference on Human Factors in Computing Systems , pages 1-19, 2026. -
  32. [36][36] Y. Chen, T. Lv, S. Zhang, Y. Yin, Y. Wan, P. S. Yu, and D. Chen. Paper2Web: Let's make your paper alive! arXiv preprint arXiv:2510.15842 , 2025. URL https://arxiv.org/abs/2510.15842 . - · arXiv
  33. [37][37] Z. Chen, J. Chen, S. O. Arik, M. Sra, T. Pfister, and J. Yoon. CoDA: Agentic systems for collaborative data visualization. arXiv preprint arXiv:2510.03194 , 2025. URL https://arxiv.org/abs/2510.03194 . - · arXiv
  34. [38][38] Z. Chen, S. Chen, Y. Ning, Q. Zhang, B. Wang, B. Yu, Y. Li, Z. Liao, C. Wei, Z. Lu, V. Dey, M. Xue, F. N. Baker, B. Burns, D. Adu-Ampratwum, X. Huang, X. Ning, S. Gao, Y. Su, and H. Sun. ScienceAgentBench: Toward rigorous assessment of language agents for data-driven scientific discovery. In International Conference on Learning Representations , 2025. URL https://arxiv.org/abs/2410.05080 . - · arXiv
  35. [39][39] Z. Chen et al. PeerCheck: Enhancing LLM-generated academic reviews towards human-level quality. arXiv preprint arXiv:2606.20897 , 2026. URL https://arxiv.org/abs/2606.20897 . - · arXiv
  36. [40][40] J. Choi, S. Park, S. Song, and H. Shim. PosterForest: Hierarchical multi-agent collaboration for scientific poster generation. arXiv preprint arXiv:2508.21720 , 2025. URL https://arxiv.org/abs/2508.21720 . - · arXiv
  37. [41][41] P. H. Couto, Q. P. Ho, N. Kumari, B. K. Rachmat, T. G. H. Khuong, I. Ullah, and L. Sun-Hosoya. RelevAIReviewer: A benchmark on AI reviewers for survey paper relevance. arXiv preprint arXiv:2406.10294 , 2024. URL https://arxiv.org/abs/2406.10294 . - · arXiv
  38. [42][42] D. Dai et al. I-WebGenBench: Evaluating interactivity in LLM-generated scientific web applications. arXiv preprint arXiv:2606.00750 , 2026. URL https://arxiv.org/abs/2606.00750 . - · arXiv
  39. [43][43] M. D'Arcy, T. Hope, L. Birnbaum, and D. Downey. MARG: Multi-agent review generation for scientific papers. arXiv preprint arXiv:2401.04259 , 2024. URL https://arxiv.org/abs/2401.04259 . - · arXiv
  40. [45][45] X. Deng, J. Da, E. Pan, Y. Y. He, C. Ide, K. Garg, N. Lauffer, A. Park, N. Pasari, C. Rane, K. Sampath, M. Krishnan, S. Kundurthy, S. Hendryx, Z. Wang, V. Bharadwaj, J. Holm, R. Aluri, C. B. C. Zhang, N. Jacobson, B. Liu, and B. Kenstler. SWE-Bench Pro: Can AI agents solve long-horizon software engineering tasks? arXiv preprint arXiv:2509.16941 , 2025. URL https://arxiv.org/abs/2509.16941 . - · arXiv
  41. [46][46] Y. Du, T. Jin, J. Kang, X. Pang, J. Chai, T. Miao, F. Liu, W. Wang, S. Yao, Y. Zhang, and S. Chen. Towards self-evolving agentic literature retrieval. arXiv preprint arXiv:2605.14306 , 2026. URL https://arxiv.org/abs/ 2605.14306 . - · arXiv
  42. [48][48] T. Fan, F. Zhang, Y. Zheng, B. Chen, X. Niu, C. Huang, J. Lin, and C. Huang. DeepInnovator: Triggering the innovative capabilities of LLMs. arXiv preprint arXiv:2602.18920 , 2026. URL https://arxiv.org/abs/2602.18920 . - · arXiv
  43. [49][49] H. Fang, Y. Feng, and I. Gurevych. From passive generation to investigation: A proactive scientific peer review agent. arXiv preprint arXiv:2606.13349 , 2026. URL https://arxiv.org/abs/2606.13349 . - · arXiv
  44. [50][50] Y. Feng, Q. Huang, X. Xie, Z. Yang, J. Yu, W. Chen, and A. K. H. Tung. IDRBench: Interactive deep research benchmark. arXiv preprint arXiv:2601.06676 , 2026. URL https://arxiv.org/abs/2601.06676 . - · arXiv
  45. [51][51] T.-J. Fu, W. Y. Wang, D. McDuff, and Y. Song. DOC2PPT: Automatic presentation slides generation from scientific documents. In AAAI Conference on Artificial Intelligence , volume 36, pages 634-642, 2022. URL https://arxiv.org/abs/2101.11796 . - · arXiv
  46. [52][52] S. K. Gaddipati, D. Muhammed, F. Keya, G. Rabby, and S. Auer. MLReplicate: Benchmarking autonomous research systems for machine learning reproducibility. arXiv preprint arXiv:2605.16616 , 2026. URL https: //arxiv.org/abs/2605.16616 . - · arXiv
  47. [53][53] S. Gao, R. Zhu, P. Sui, Z. Kong, S. Aldogom, Y. Huang, A. Noori, R. Shamji, K. Parvataneni, T. Tsiligkaridis, and M. Zitnik. Democratizing AI scientists using ToolUniverse. arXiv preprint arXiv:2509.23426 , 2025. URL https://arxiv.org/abs/2509.23426 . - · arXiv
  48. [54][54] S. Gao et al. AutoScientists: Self-organizing agent teams for long-running scientific experimentation. arXiv preprint arXiv:2605.28655 , 2026. URL https://arxiv.org/abs/2605.28655 . - · arXiv
  49. [55][55] X. Gao, J. Ruan, Z. Zhang, J. Gao, T. Liu, and Y. Fu. ReviewAgents: Bridging the gap between human and AI-generated paper reviews. arXiv preprint arXiv:2503.08506 , 2025. URL https://arxiv.org/abs/2503.08506 . - · arXiv
  50. [56][56] Y. Gao, Q. Wu, and L. Zhu. Merging the citations received by arXiv-deposited e-prints and their corresponding published journal articles: Problems and perspectives. Information Processing & Management , 57(5):102267, 2020. -
  51. [57][57] Z. Gao, K. Brantley, and T. Joachims. Reviewer2: Optimizing review generation through prompt generation. arXiv preprint arXiv:2402.10886 , 2024. URL https://arxiv.org/abs/2402.10886 . - · arXiv
  52. [58][58] K. Garg, F. Shaik, S. Bandyopadhyay, and C. Caragea. Let's use ChatGPT to write our paper! benchmarking LLMs to write the introduction of a research paper. arXiv preprint arXiv:2508.14273 , 2025. URL https: //arxiv.org/abs/2508.14273 . - · arXiv
  53. [59][59] A. Garikaparthi, M. Patwardhan, L. Vig, and A. Cohan. IRIS: Interactive research ideation system for accelerating scientific discovery. In Annual Meeting of the Association for Computational Linguistics , pages 592-603, 2025. URL https://arxiv.org/abs/2504.16728 . - · arXiv
  54. [60][60] J. Ge, Z. Z. Wang, X. Zhou, Y.-H. Peng, S. Subramanian, Q. Tan, M. Sap, A. Suhr, D. Fried, G. Neubig, and T. Darrell. AutoPresent: Designing structured visuals from scratch. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2902-2911, 2025. URL https://arxiv.org/abs/2501.00912 . - · arXiv
  55. [61][61] A. Ghafarollahi and M. J. Buehler. SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoning. arXiv preprint arXiv:2409.05556 , 2024. URL https://arxiv.org/abs/2409.05556 . - · arXiv
  56. [62][62] E. Gibney. Major conference catches illicit AI use - and rejects hundreds of papers. Nature News , 652:281-282, 2026. -
  57. [63][63] G. H. T. Go, K. Ly, A. Sogaard, A. Tabatabaei, M. de Rijke, and X. Chen. LiRA: A multi-agent framework for reliable and readable literature review generation. arXiv preprint arXiv:2510.05138 , 2025. URL https: //arxiv.org/abs/2510.05138 . - · arXiv
  58. [64][64] S. Goel, R. Hazra, D. Jayalath, T. Willi, P. Jain, W. F. Shen, I. Leontiadis, F. Barbieri, Y. Bachrach, J. Geiping, and C. Whitehouse. Training AI co-scientists using rubric rewards. arXiv preprint arXiv:2512.23707 , 2025. URL https://arxiv.org/abs/2512.23707 . - · arXiv
  59. [65][65] K. Goswami, P. Mathur, R. Rossi, and F. Dernoncourt. PlotGen: Multi-agent LLM-based scientific data visualization via multimodal feedback. arXiv preprint arXiv:2502.00988 , 2025. URL https://arxiv.org/abs/ 2502.00988 . - · arXiv
  60. [66][66] J. Gottweis, W.-H. Weng, A. Daryin, T. Tu, A. Palepu, P. Sirkovic, A. Myaskovsky, F. Weissenberger, K. Rong, R. Tanno, K. Saab, D. Popovici, J. Blum, F. Zhang, K. Chou, A. Hassidim, B. Gokturk, A. Vahdat, P. Kohli, Y. Matias, A. Carroll, K. Kulkarni, N. Tomasev, Y. Guan, V. Dhillon, E. D. Vaishnav, B. Lee, T. R. D. Costa, J. R. Penadés, G. Peltz, Y. Xu, A. Pawlosky, A. Karthikesalingam, and V. Natarajan. Towards an AI co-scientist. arXiv preprint arXiv:2502.18864 , 2025. URL https://arxiv.org/abs/2502.18864 . - · arXiv
  61. [67][67] P. Goyal, M. Parmar, Y. Song, H. Palangi, T. Pfister, and J. Yoon. ScholarPeer: A context-aware multi-agent framework for automated peer review. arXiv preprint arXiv:2601.22638 , 2026. URL https://arxiv.org/abs/2601. 22638 . - · arXiv
  62. [68][68] C. Greisinger and S. Eger. TikZilla: Scaling text-to-TikZ with high-quality data and reinforcement learning. arXiv preprint arXiv:2603.03072 , 2026. URL https://arxiv.org/abs/2603.03072 . - · arXiv
  63. [69][69] T. Gu, J. Wang, Z. Zhang, and H. Li. LLMs can realize combinatorial creativity: Generating creative ideas via LLMs for scientific research. arXiv preprint arXiv:2412.14141 , 2024. URL https://arxiv.org/abs/2412.14141 . - · arXiv
  64. [70][70] S. Guo, C. Deng, Y. Wen, H. Chen, Y. Chang, and J. Wang. DS-Agent: Automated data science by empowering large language models with case-based reasoning. arXiv preprint arXiv:2402.17453 , 2024. URL https://arxiv. org/abs/2402.17453 . - · arXiv
  65. [71][71] S. Guo, A. H. Shariatmadari, G. Xiong, A. Huang, M. Kim, C. M. Williams, S. Bekiranov, and A. Zhang. IdeaBench: Benchmarking large language models for research idea generation. In ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 5888-5899, 2025. URL https://arxiv.org/abs/2411.02429 . - · arXiv
  66. [72][72] B. Gyevnár et al. Distributed denial of science: How indirect data poisoning of AI systems can industrialize scientific fraud. arXiv preprint arXiv:2607.10712 , 2026. URL https://arxiv.org/abs/2607.10712 . - · arXiv
  67. [73][73] P. Han, Y. Yu, J. Xu, and J. You. DRPG (decompose, retrieve, plan, generate): An agentic framework for academic rebuttal. arXiv preprint arXiv:2601.18081 , 2026. URL https://arxiv.org/abs/2601.18081 . - · arXiv
  68. [75][75] Y. He, G. Huang, P. Feng, Y. Lin, Y. Zhang, H. Li, and W. E. PaSa: An LLM agent for comprehensive academic paper search. arXiv preprint arXiv:2501.10120 , 2025. URL https://arxiv.org/abs/2501.10120 . - · arXiv
  69. [76][76] Z. He, Z. Lyu, and Y. R. Fung. RebuttalAgent: Strategic persuasion in academic rebuttal via theory of mind. arXiv preprint arXiv:2601.15715 , 2026. URL https://arxiv.org/abs/2601.15715 . - · arXiv
  70. [78][78] S.-T. Ho, M. Liu, H. Nghiem, and F. Huang. SoundnessBench: Can your AI scientist really tell good research ideas from bad ones? arXiv preprint arXiv:2605.30329 , 2026. URL https://arxiv.org/abs/2605.30329 . - · arXiv
  71. [79][79] M. Hong, D. Jiang, C. J. Zhang, Z. Guo, Y. Li, J. Chen, S. Cui, and Z. Su. CiteLLM: An agentic platform for trustworthy scientific reference discovery. arXiv preprint arXiv:2602.23075 , 2026. URL https://arxiv.org/abs/ 2602.23075 . - · arXiv
  72. [80][80] E. Hossain, S. K. Sinha, N. Bansal, R. A. Knipper, S. Sarkar, J. Salvador, Y. Mahajan, S. R. P. K. Guttikonda, M. Akter, M. M. Hassan, M. Freestone, M. C. W. Jr., D. Feng, and S. Karmaker. LLMs as meta-reviewers' assistants: A case study. In Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics , pages 7763-7803, 2025. URL https://arxiv.org/abs/2402.15589 . - · arXiv
  73. [81][81] J. Hou, A. H. Lin, N. Chen, Y. Gong, and B. He. PaperDebugger: A plugin-based multi-agent system for in-editor academic writing, review, and editing. arXiv preprint arXiv:2512.02589 , 2025. URL https: //arxiv.org/abs/2512.02589 . - · arXiv
  74. [82][82] C.-C. Hsu, E. Bransom, J. Sparks, B. Kuehl, C. Tan, D. Wadden, L. L. Wang, and A. Naik. CHIME: LLM-assisted hierarchical organization of scientific studies for literature review support. arXiv preprint arXiv:2407.16148 , 2024. URL https://arxiv.org/abs/2407.16148 . - · arXiv
  75. [83][83] X. Hu, H. Fu, J. Wang, Y. Wang, Z. Li, R. Xu, Y. Lu, Y. Jin, L. Pan, and Z. Lan. Nova: An iterative planning and search approach to enhance novelty and diversity of LLM generated ideas. arXiv preprint arXiv:2410.14255 , 2024. URL https://arxiv.org/abs/2410.14255 . - · arXiv
  76. [84][84] X. Hu, Z. Zhao, S. Wei, Z. Chai, Q. Ma, G. Wang, X. Wang, J. Su, J. Xu, M. Zhu, Y. Cheng, J. Yuan, J. Li, K. Kuang, Y. Yang, H. Yang, and F. Wu. InfiAgent-DABench: Evaluating agents on data analysis tasks. arXiv preprint arXiv:2401.05507 , 2024. URL https://arxiv.org/abs/2401.05507 . - · arXiv
  77. [85][85] T. Hua, H. Hua, V. Xiang, B. Klieger, S. T. Truong, W. Liang, F.-Y. Sun, and N. Haber. ResearchCodeBench: Benchmarking LLMs on implementing novel machine learning research code. arXiv preprint arXiv:2506.02314 , 2025. URL https://arxiv.org/abs/2506.02314 . - · arXiv
  78. [86][86] E. Huang et al. NVAITC AI Scientist: A governed end-to-end research system - a hypertension GWAS case study. arXiv preprint arXiv:2607.11084 , 2026. URL https://arxiv.org/abs/2607.11084 . - · arXiv
  79. [88][88] Q. Huang, J. Vora, P. Liang, and J. Leskovec. MLAgentBench: Evaluating language agents on machine learning experimentation. In International Conference on Machine Learning , 2024. URL https://arxiv.org/abs/2310. 03302 . -
  80. [89][89] S. Huang, Y. Gao, J. Bai, Y. Zhou, Z. Yin, X. Liu, R. Chellappa, C. P. Lau, S. Nag, C. Peng, and S. Pramanick. SciFig: Towards automating scientific figure generation. arXiv preprint arXiv:2601.04390 , 2026. URL https: //arxiv.org/abs/2601.04390 . - · arXiv
  81. [90][90] K. Hur et al. Compressing the validation bottleneck: An agentic self-driving lab for scientific discovery. arXiv preprint arXiv:2607.04508 , 2026. URL https://arxiv.org/abs/2607.04508 . - · arXiv
  82. [91][91] M. Idahl and Z. Ahmadi. OpenReviewer: A specialized large language model for generating critical scientific paper reviews. In Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies , pages 550-562, 2025. URL https://arxiv.org/abs/2412.11948 . - · arXiv
  83. [92][92] A. K. Jagadish et al. Closing the loop to discover psychological theories with an automated cognitive scientist. arXiv preprint arXiv:2606.26448 , 2026. URL https://arxiv.org/abs/2606.26448 . - · arXiv
  84. [93][93] P. Jansen, M.-A. Cote, T. Khot, E. Bransom, B. Dalvi Mishra, B. P. Majumder, O. Tafjord, and P. Clark. DiscoveryWorld: A virtual environment for developing and evaluating automated scientific discovery agents. In Advances in Neural Information Processing Systems , 2024. URL https://arxiv.org/abs/2406.06769 . - · arXiv
  85. [94][94] P. Jansen, O. Tafjord, M. Radensky, P. Siangliulue, T. Hope, B. Dalvi Mishra, B. P. Majumder, D. S. Weld, and P. Clark. CodeScientist: End-to-end semi-automated scientific discovery with code-based experimentation. In Annual Meeting of the Association for Computational Linguistics , pages 13370-13467, 2025. URL https: //arxiv.org/abs/2503.22708 . -
  86. [95][95] B. Jiang. HindSight: Evaluating LLM-generated research ideas via future impact. arXiv preprint arXiv:2603.15164 , 2026. URL https://arxiv.org/abs/2603.15164 . - · arXiv
  87. [96][96] L. Jiang, Y. Chai, M. Li, M. Liu, R. Fok, N. Dziri, Y. Tsvetkov, M. Sap, A. Albalak, and Y. Choi. Artificial hivemind: The open-ended homogeneity of language models (and beyond). In Advances in Neural Information Processing Systems , 2025. URL https://arxiv.org/abs/2510.22954 . - · arXiv
  88. [98][98] Z. Jiang, D. Schmidt, D. Srikanth, D. Xu, I. Kaplan, D. Jacenko, and Y. Wu. AIDE: AI-driven exploration in the space of code. arXiv preprint arXiv:2502.13138 , 2025. URL https://arxiv.org/abs/2502.13138 . - · arXiv
  89. [99][99] C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan. SWE-bench: Can language models resolve real-world GitHub issues? In International Conference on Learning Representations , 2024. URL https://arxiv.org/abs/2310.06770 . - · arXiv
  90. [100][100] J. Jin, Y. Hu, K. Qiu, Q. Dai, C. Luo, G. Dong, X. Li, T. Zhao, X. Ma, G. Zhang, Z. Wu, B. Liu, Z. Yang, L. Li, L. Wang, H. Qian, Y. Zhu, and Z. Dou. Toward generalist autonomous research via hypothesis-tree refinement. arXiv preprint arXiv:2606.11926 , 2026. URL https://arxiv.org/abs/2606.11926 . - · arXiv
  91. [101][101] Y. Jin, Q. Zhao, Y. Wang, H. Chen, K. Zhu, Y. Xiao, and J. Wang. AgentReview: Exploring peer review dynamics with LLM agents. In Conference on Empirical Methods in Natural Language Processing , pages 1208-1226, 2024. URL https://arxiv.org/abs/2406.12708 . - · arXiv
  92. [102][102] K. Jung, H. Cho, J. Yun, S. Yang, J. Jang, and J. Choo. Talk to your slides: High-efficiency slide editing via language-driven structured data manipulation. arXiv preprint arXiv:2505.11604 , 2025. URL https://arxiv.org/ abs/2505.11604 . - · arXiv
  93. [103][103] S. Kale and V. Nadadur. TeXpert: A multi-level benchmark for evaluating LaTeX code generation by LLMs. In Workshop on Scholarly Document Processing , pages 7-16, 2025. URL https://arxiv.org/abs/2506.16990 . - · arXiv
  94. [104][104] A. H. Kargaran, N. Nikeghbal, J. Yang, and N. Ousidhoum. Insights from the ICLR peer review and rebuttal process. arXiv preprint arXiv:2511.15462 , 2025. URL https://arxiv.org/abs/2511.15462 . - · arXiv
  95. [106][106] J. Keuper. Prompt injection attacks on LLM generated reviews of scientific publications. arXiv preprint arXiv:2509.10248 , 2025. URL https://arxiv.org/abs/2509.10248 . - · arXiv
  96. [107][107] J. Khatri and M. Patwardhan. Defend: Automated rebuttals for peer review with minimal author guidance. arXiv preprint arXiv:2603.27360 , 2026. URL https://arxiv.org/abs/2603.27360 . - · arXiv
  97. [108][108] S. Kim, D. Yoon, K. Gashteovski, J. Suk, J. Baek, P. Aggarwal, I. Wu, V. Zaverkin, S. Petkoski, D. R. Schrider, I. Dukovski, F. Santini, B. Mitreska, Y. Jeong, K. Kwon, Y. M. Sim, D. Manasova, A. Porto, B. Mojsoska, M. Takamoto, M. Shuntov, R. Liu, H. J. Lee, N. U. Dinç, Y. Jo, S. Han, C. Lee, H. Li, E. H. R. Tsai, E. Simsek, K. Shafi, Y. Chung, J. Park, A. Shulevski, H. Christiansen, Y. Son, E. Knight, A. Montoya, J. Ahn, C. Langkammer, H. Moon, C. Yoon, N. Stikov, M. Jang, E. Choi, J. Kim, Y. S. Jung, W. Y. Kim, J. K. Kim, I. M. Anjum, H. U. Kim, D. Bridges, C. Lawrence, X. Yue, A. Oh, A. Asai, S. Welleck, and G. Neubig. On the limits and opportunities of AI reviewers: Reviewing the reviews of nature-family papers with 45 expert scientists. arXiv preprint arXiv:2605.20668 , 2026. URL https://arxiv.org/abs/2605.20668 . - · arXiv
  98. [109][109] Y. Kim, K. Gu, C. Park, C. Park, S. Schmidgall, A. A. Heydari, Y. Yan, Z. Zhang, Y. Zhuang, M. Malhotra, P. P. Liang, H. W. Park, Y. Yang, X. Xu, Y. Du, S. Patel, T. Althoff, D. McDuff, and X. Liu. Towards a science of scaling agent systems. arXiv preprint arXiv:2512.08296 , 2025. URL https://arxiv.org/abs/2512.08296 . - · arXiv
  99. [110][110] D. Kobak, R. González-Márquez, E. Ákos Bernárdez-Vilaboa, and J. Lause. Delving into LLM-assisted writing in biomedical publications through excess vocabulary. Science Advances , 11(27):eadt3813, 2025. URL https://arxiv.org/abs/2406.07016 . - · arXiv
  100. [111][111] P. T. J. Kon, J. Liu, Q. Ding, Y. Qiu, Z. Yang, Y. Huang, J. Srinivasa, M. Lee, M. Chowdhury, and A. Chen. Curie: Toward rigorous and automated scientific experimentation with AI agents. arXiv preprint arXiv:2502.16069 , 2025. URL https://arxiv.org/abs/2502.16069 . - · arXiv
  101. [112][112] P. T. J. Kon, Q. Ding, J. Liu, X. Zhu, J. Peng, J. Xing, Y. Huang, Y. Qiu, J. Srinivasa, M. Lee, M. Chowdhury, M. Zaharia, and A. Chen. EXP-Bench: Can AI conduct AI research experiments? In International Conference on Learning Representations , 2026. URL https://arxiv.org/abs/2505.24785 . - · arXiv
  102. [113][113] K. Kusumegi, X. Yang, P. Ginsparg, M. de Vaan, T. Stuart, and Y. Yin. Scientific production in the era of large language models. Science , 390:1240-1243, 2025. URL https://arxiv.org/abs/2601.13187 . - · arXiv
  103. [114][114] R. T. Lange, Y. Imajuku, and E. Cetin. ShinkaEvolve: Towards open-ended and sample-efficient program evolution. arXiv preprint arXiv:2509.19349 , 2025. URL https://arxiv.org/abs/2509.19349 . - · arXiv
  104. [115][115] J. M. Laurent, J. D. Janizek, M. Ruzo, M. M. Hinks, M. J. Hammerling, S. Narayanan, M. Ponnapati, A. D. White, and S. G. Rodriques. Lab-Bench: Measuring capabilities of language models for biology research. arXiv preprint arXiv:2407.10362 , 2024. URL https://arxiv.org/abs/2407.10362 . - · arXiv
  105. [116][116] C. Leblanc et al. To trust or not to trust: Authors' response to AI-based reviews. arXiv preprint arXiv:2605.16623 , 2026. URL https://arxiv.org/abs/2605.16623 . - · arXiv
  106. [117][117] M. Lee, P. Liang, and Q. Yang. CoAuthor: Human-AI collaborative writing with language models. arXiv preprint arXiv:2201.06796 , 2022. URL https://arxiv.org/abs/2201.06796 . - · arXiv
  107. [118][118] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, S. Riedel, and D. Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems , volume 33, pages 9459-9474, 2020. URL https://arxiv.org/abs/2005.11401 . - · arXiv
  108. [120][120] E. Li, J. Ren, X. Pan, C. Yan, C. Li, D. Bergemann, and Z. Yang. Build your personalized research group: A multiagent framework for continual and interactive science automation. arXiv preprint arXiv:2510.15624 , 2025. URL https://arxiv.org/abs/2510.15624 . - · arXiv
  109. [121][121] J. Li, S. Li, Z. Gao, Q. Shi, Y. Li, Z. Wang, J. Huang, H. Wang, J. Wang, X. Han, Z. Liu, and M. Sun. TritonBench: Benchmarking large language model capabilities for generating triton operators. In Annual Meeting on Association for Computational Linguistics , 2025. URL https://arxiv.org/abs/2502.14752 . - · arXiv
  110. [122][122] L. Li, W. Xu, J. Guo, R. Zhao, X. Li, Y. Yuan, B. Zhang, Y. Jiang, Y. Xin, R. Dang, D. Zhao, Y. Rong, T. Feng, and L. Bing. Chain of ideas: Revolutionizing research via novel idea development with LLM agents. arXiv preprint arXiv:2410.13185 , 2024. URL https://arxiv.org/abs/2410.13185 . - · arXiv
  111. [123][123] L. Li et al. Gaming AI-assisted peer reviews poses new risks to the scientific community. arXiv preprint arXiv:2606.10159 , 2026. URL https://arxiv.org/abs/2606.10159 . - · arXiv
  112. [124][124] M. Li, Y. Zeng, Z. Cheng, C. Ma, and K. Jia. ReportBench: Evaluating deep research agents via academic survey tasks. arXiv preprint arXiv:2508.15804 , 2025. URL https://arxiv.org/abs/2508.15804 . - · arXiv
  113. [125][125] M. Li, Z. Lin, and S. Ma. Source or it didn't happen: A multi-agent framework for citation hallucination detection. arXiv preprint arXiv:2605.08583 , 2026. URL https://arxiv.org/abs/2605.08583 . - · arXiv
  114. [126][126] R. Li, T. Patel, Q. Wang, and X. Du. MLR-Copilot: Autonomous machine learning research based on large language models agents. arXiv preprint arXiv:2408.14033 , 2024. URL https://arxiv.org/abs/2408.14033 . - · arXiv
  115. [127][127] S. Li, Y. Cai, H. Chen, and Y. Wang. GeoSVG-RL: Geometry-aware reinforcement learning for layout-constrained text-to-SVG diagram generation. arXiv preprint arXiv:2605.25447 , 2026. URL https://arxiv.org/abs/2605. 25447 . - · arXiv
  116. [128][128] S. Li, L. Fan, Y. Lin, Z. Li, X. Wei, S. Ni, H. Alinejad-Rokny, and M. Yang. Automatic paper reviewing with heterogeneous graph reasoning over LLM-simulated reviewer-author debates. arXiv preprint arXiv:2511.08317 , 2026. URL https://arxiv.org/abs/2511.08317 . AAAI-2026. - · arXiv
  117. [130][130] Y. Li, C. Shao, X. Liu, R. Zhao, P. Liu, H. Su, Z. Chen, Q. Yang, A. Xu, Y. Fang, Q. Zeng, T. Li, J. Xu, F. Xu, Y. Li, and T.-Y. Liu. AutoSOTA: An end-to-end automated research system for state-of-the-art AI model discovery. arXiv preprint arXiv:2604.05550 , 2026. URL https://arxiv.org/abs/2604.05550 . - · arXiv
  118. [131][131] Z. Li, D. Jiang, X. Ma, H. Zhang, P. Nie, Y. Zhang, K. Zou, J. Xie, Y. Zhang, and W. Chen. OpenResearcher: A fully open pipeline for long-horizon deep research trajectory synthesis. arXiv preprint arXiv:2603.20278 , 2026. URL https://arxiv.org/abs/2603.20278 . - · arXiv
  119. [132][132] W. Liang, Y. Zhang, Z. Wu, H. Lepp, W. Ji, X. Zhao, H. Cao, S. Liu, S. He, Z. Huang, D. Yang, C. Potts, C. D. Manning, and J. Y. Zou. Mapping the increasing use of LLMs in scientific papers. arXiv preprint arXiv:2404.01268 , 2024. URL https://arxiv.org/abs/2404.01268 . - · arXiv
  120. [133][133] X. Liang, J. Yang, Y. Wang, C. Tang, Z. Zheng, S. Song, Z. Lin, Y. Yang, S. Niu, H. Wang, B. Tang, F. Xiong, K. Mao, and Z. Li. SurveyX: Academic survey automation via large language models. arXiv preprint arXiv:2502.14776 , 2025. URL https://arxiv.org/abs/2502.14776 . - · arXiv
  121. [134][134] X. Liang, X. Zhang, Y. Xu, S. Sun, and C. You. SlideGen: Collaborative multimodal agents for scientific slide generation. arXiv preprint arXiv:2512.04529 , 2025. URL https://arxiv.org/abs/2512.04529 . - · arXiv
  122. [135][135] X. Liao, L. Li, H. Liu, T. Gu, Y. Yao, T. Zhu, Y. Teng, and Y. Wang. PseudoBench: Measuring how agentic autoresearch fuels pseudoscience. arXiv preprint arXiv:2606.18060 , 2026. URL https://arxiv.org/abs/2606.18060 . - · arXiv
  123. [136][136] C.-Y. Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out , pages 74-81, 2004. -
  124. [137][137] T.-L. Lin, W.-C. Chen, T.-F. Hsiao, H.-I. Liu, Y.-H. Yeh, Y.-K. Chan, W.-S. Lien, P.-Y. Kuo, P. S. Yu, and H.-H. Shuai. Breaking the reviewer: Assessing the vulnerability of large language models in automated peer review under textual adversarial attacks. arXiv preprint arXiv:2506.11113 , 2025. URL https://arxiv.org/abs/2506.11113 . - · arXiv
  125. [138][138] Z. Lin, Q. Xie, M. Zhu, S. Li, Q. Sun, E. Gu, Y. Ding, K. Sun, F. Guo, P. Lu, Z. Ning, Y. Weng, and Y. Zhang. AutoFigure-Edit: Generating editable scientific illustration. arXiv preprint arXiv:2603.06674 , 2026. URL https://arxiv.org/abs/2603.06674 . - · arXiv
  126. [139][139] J. Ling, Y. Qi, T. Huang, S. Zhou, Y. Huang, J. Yang, Z. Song, Y. Zhou, Y. Yang, H. T. Shen, and P. Wang. Table2LaTeX-RL: High-fidelity LaTeX code generation from table images via reinforced multimodal language models. arXiv preprint arXiv:2509.17589 , 2025. URL https://arxiv.org/abs/2509.17589 . - · arXiv
  127. [140][140] J. Liu, L. Yang, H. Luo, F. Wang, H. Li, and M. Wang. Preacher: Paper-to-video agentic system. In IEEE/CVF International Conference on Computer Vision , pages 17129-17139, 2025. URL https://arxiv.org/abs/2508.09632 . - · arXiv
  128. [141][141] J. Liu, P. Xia, S. Han, S. Qiu, L. Zhang, G. Chen, H. Tu, X. Yang, J. Zhou, H. Zhu, Y. Li, J. Zhang, Y. Zhou, Z. Zheng, C. Xie, M. Ding, and H. Yao. AutoResearchClaw: Self-reinforcing autonomous research with humanai collaboration. https://github.com/aiming-lab/AutoResearchClaw , 2026. URL https://arxiv.org/abs/2605. 20025 . -
  129. [142][142] J. Liu, T. J. Zhang, R. Faulkner, X. A. Huang, V. Zouhar, D. Glandorf, I. Dahlgren, V. Q. Truong, R. Dagli, Y. Chen, F. Leeb, P. S. Pandey, Y. Bicker, S. Majumder, W. Jiang, Z. Qiu, S. P. Chowdhury, B. Schölkopf, M. Diab, and Z. Jin. PaperMentor: A human-centered multi-agent writing tutor for AI research papers on overleaf. arXiv preprint arXiv:2606.08857 , 2026. URL https://arxiv.org/abs/2606.08857 . - · arXiv
  130. [143][143] W. Liu, Z. Yang, Y. Zhao, and X. Li. RATE: Reviewer profiling and annotation-free training for expertise ranking in peer review systems. arXiv preprint arXiv:2601.19637 , 2026. URL https://arxiv.org/abs/2601.19637 . - · arXiv
  131. [144][144] Y. Liu, Y. Wu, D. Zhang, and L. Sun. Agentic AutoSurvey: Let LLMs survey LLMs. arXiv preprint arXiv:2509.18661 , 2025. URL https://arxiv.org/abs/2509.18661 . - · arXiv
  132. [145][145] Y. Liu, Z. Yang, T. Xie, J. Ni, B. Gao, Y. Li, S. Tang, W. Ouyang, E. Cambria, and D. Zhou. ResearchBench: Benchmarking LLMs in scientific discovery via inspiration-based task decomposition. In Findings of the Annual Meeting of the Association for Computational Linguistics , 2026. URL https://arxiv.org/abs/2503.21248 . - · arXiv
  133. [146][146] Z. Liu, X. Bao, P. Li, J. Zhou, Z. Liao, Y. He, K. Jiang, C.-W. Xie, Y. Zheng, and H. Xie. ShowTable: Unlocking creative table visualization with collaborative reflection and refinement. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2026. URL https://arxiv.org/abs/2512.13303 . - · arXiv
  134. [147][147] N. P. P. Loc, T. H. L. Viet, T. T. Khanh, D. A. Nguyen, T. A. N. Pham, T. Nguyen, N. V. Chawla, W. Buntine, K.-S. Wong, K. D. Doan, and B. T. Nguyen. PRISM: A multi-dimensional benchmark for evaluating LLM peer reviewers. arXiv preprint arXiv:2605.26730 , 2026. URL https://arxiv.org/abs/2605.26730 . - · arXiv
  135. [148][148] M. Louis et al. Rebuttals move peer-review scores, but initial-review structure bounds the movement. arXiv preprint arXiv:2606.22166 , 2026. URL https://arxiv.org/abs/2606.22166 . - · arXiv
  136. [149][149] C. Lu, C. Lu, R. T. Lange, J. Foerster, J. Clune, and D. Ha. The AI scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292 , 2024. URL https://arxiv.org/abs/2408.06292 . - · arXiv
  137. [152][152] Z. Luo, A. Kasirzadeh, and N. B. Shah. The more you automate, the less you see: Hidden pitfalls of AI scientist systems. arXiv preprint arXiv:2509.08713 , 2025. URL https://arxiv.org/abs/2509.08713 . - · arXiv
  138. [153][153] Z. Luo, Z. Yang, Z. Xu, W. Yang, and X. Du. LLM4SR: A survey on large language models for scientific research. arXiv preprint arXiv:2501.04306 , 2025. URL https://arxiv.org/abs/2501.04306 . - · arXiv
  139. [154][154] Y. Lyu, X. Zhang, X. Yi, Y. Zhao, S. Guo, W. Hu, J. Piotrowski, J. Kaliski, J. Urbani, Z. Meng, L. Zhou, and X. Yan. EvoScientist: Towards multi-agent evolving AI scientists for end-to-end scientific discovery. arXiv preprint arXiv:2603.08127 , 2026. URL https://arxiv.org/abs/2603.08127 . - · arXiv
  140. [155][155] J. Lála, O. O'Donoghue, A. Shtedritski, S. Cox, S. G. Rodriques, and A. D. White. PaperQA: Retrieval-augmented generative agent for scientific research. arXiv preprint arXiv:2312.07559 , 2023. URL https://arxiv.org/abs/ 2312.07559 . - · arXiv
  141. [156][156] M. M. K. M, V. N. Balasubramanian, and C. V. Jawahar. Unifying scientific communication: Fine-grained correspondence across scientific media. arXiv preprint arXiv:2605.05831 , 2026. URL https://arxiv.org/abs/ 2605.05831 . - · arXiv
  142. [157][157] Q. Ma, C. Guo, Z. Tian, S. Wang, J. Xiao, Y. Yue, and Z. Zhang. Paper2Rebuttal: A multi-agent framework for transparent author response assistance. arXiv preprint arXiv:2601.14171 , 2026. URL https://arxiv.org/abs/ 2601.14171 . - · arXiv
  143. [158][158] B. P. Majumder, H. Surana, D. Agarwal, B. D. Mishra, A. Meena, A. Prakhar, T. Vora, T. Khot, A. Sabharwal, and P. Clark. DiscoveryBench: Towards data-driven discovery with large language models. In International Conference on Learning Representations , 2025. URL https://arxiv.org/abs/2407.01725 . - · arXiv
  144. [159][159] R. Meng, B. D. Mishra, J. Chen, C.-L. Li, P. Goyal, M. Parmar, Y. Song, Y. Song, R. Sinha, P. Ranganathan, B. Gokturk, J. Yoon, and T. Pfister. ScientistOne: Towards human-level autonomous research via chain-ofevidence. arXiv preprint arXiv:2605.26340 , 2026. URL https://arxiv.org/abs/2605.26340 . - · arXiv
  145. [161][161] J. Miao, J. R. Davis, Y. Zhang, J. K. Pritchard, and J. Zou. Paper2Agent: Reimagining research papers as interactive and reliable AI agents. arXiv preprint arXiv:2509.06917 , 2025. URL https://arxiv.org/abs/2509. 06917 . - · arXiv
  146. [162][162] L. Mitchener, A. Yiu, B. Chang, M. Bourdenx, T. Nadolski, A. Sulovari, E. C. Landsness, D. L. Barabasi, S. Narayanan, N. Evans, S. Reddy, M. Foiani, A. Kamal, L. P. Shriver, F. Cao, A. T. Wassie, J. M. Laurent, E. Melville-Green, M. Caldas, A. Bou, K. F. Roberts, S. Zagorac, T. C. Orr, M. E. Orr, K. J. Zwezdaryk, A. E. Ghareeb, L. McCoy, B. Gomes, E. A. Ashley, K. E. Duff, T. Buonassisi, T. Rainforth, R. J. Bateman, M. Skarlinski, S. G. Rodriques, M. M. Hinks, and A. D. White. Kosmos: An AI scientist for autonomous discovery. arXiv preprint arXiv:2511.02824 , 2025. URL https://arxiv.org/abs/2511.02824 . - · arXiv
  147. [163][163] I. Mondal et al. A good talk does not look like a summary, it teaches you! measuring takeaways from paper-to-video talks. arXiv preprint arXiv:2606.28531 , 2026. URL https://arxiv.org/abs/2606.28531 . - · arXiv
  148. [164][164] J. Mun, C. Jung, X. Zhou, H. Kim, and M. Sap. GoodPoint: Learning constructive scientific paper feedback from author responses. arXiv preprint arXiv:2604.11924 , 2026. URL https://arxiv.org/abs/2604.11924 . - · arXiv
  149. [165][165] M. Naddaf. More than half of researchers now use AI for peer review - often against guidance. Nature News , 649:273-274, 2026. -
  150. [166][166] D. Nathani, L. Madaan, N. Roberts, N. Bashlykov, A. Menon, V. Moens, A. Budhiraja, D. Magka, V. Vorotilov, G. Chaurasia, D. Hupkes, R. S. Cabral, T. Shavrina, J. Foerster, Y. Bachrach, W. Y. Wang, and R. Raileanu. MLGym: A new framework and benchmark for advancing AI research agents. arXiv preprint arXiv:2502.14499 , 2025. URL https://arxiv.org/abs/2502.14499 . - · arXiv
  151. [167][167] B. Newman, Y. Lee, A. Naik, P. Siangliulue, R. Fok, J. Kim, D. S. Weld, J. C. Chang, and K. Lo. ArxivDIGESTables: Synthesizing scientific literature into tables using language models. In Conference on Empirical Methods in Natural Language Processing , pages 9612-9631, 2024. URL https://arxiv.org/abs/2410.22360 . - · arXiv
  152. [168][168] M.-A. Nguyen, M.-D. Nguyen, H. L. N.T., K. H. Dang, N. T. Dong, and D. D. Le. SurveyG: A multi-agent LLM framework with hierarchical citation graph for automated survey generation. arXiv preprint arXiv:2510.07733 , 2025. URL https://arxiv.org/abs/2510.07733 . - · arXiv
  153. [170][170] Y. Ni, P. Nie, K. Zou, X. Yue, and W. Chen. VisCoder: Fine-tuning LLMs for executable python visualization code generation. In Conference on Empirical Methods in Natural Language Processing , pages 2956-2983, 2025. URL https://arxiv.org/abs/2506.03930 . - · arXiv
  154. [171][171] J. Ning et al. Closed-loop auto research for molecular property prediction: Discovering and certifying generalizable improvements. arXiv preprint arXiv:2606.22731 , 2026. URL https://arxiv.org/abs/2606.22731 . - · arXiv
  155. [172][172] A. Novikov, N. Vu, M. Eisenberger, E. Dupont, P.-S. Huang, A. Z. Wagner, S. Shirobokov, B. Kozlovskii, F. J. R. Ruiz, A. Mehrabian, M. P. Kumar, A. See, S. Chaudhuri, G. Holland, A. Davies, S. Nowozin, P. Kohli, and M. Balog. AlphaEvolve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131 , 2025. URL https://arxiv.org/abs/2506.13131 . - · arXiv
  156. [173][173] O. O'Donoghue, A. Shtedritski, J. Ginger, R. Abboud, A. E. Ghareeb, J. Booth, and S. G. Rodriques. BioPlanner: Automatic evaluation of LLMs on protocol planning. arXiv preprint arXiv:2310.10632 , 2023. URL https: //arxiv.org/abs/2310.10632 . - · arXiv
  157. [174][174] J. Ou, W. G. Walden, K. Sanders, Z. Jiang, K. Sun, J. Cheng, W. Jurayj, M. Wanner, S. Liang, C. Morgan, S. Han, W. Wang, C. May, H. Recknor, D. Khashabi, and B. V. Durme. ClaimCheck: How grounded are LLM critiques of scientific papers? In Conference on Empirical Methods in Natural Language Processing , pages 21712-21735, 2025. URL https://arxiv.org/abs/2503.21717 . - · arXiv
  158. [175][175] A. Ouyang, S. Guo, S. Arora, A. L. Zhang, W. Hu, C. Ré, and A. Mirhoseini. KernelBench: Can LLMs write efficient GPU kernels? In International Conference on Machine Learning , 2025. URL https://arxiv.org/abs/ 2502.10517 . -
  159. [176][176] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems , volume 35, pages 27730-27744, 2022. URL https://arxiv.org/abs/2203.02155 . - · arXiv
  160. [177][177] B. Pan, Y. Fu, K. Wang, J. Lu, L. Pan, Z. Qian, Y. Chen, G. Wang, Y. Zhou, L. Zheng, Y. Tang, Z. Wen, Y. Wu, J. Lu, B. Zhu, M. Zhu, B. Zhang, and W. Chen. VIS-Shepherd: Constructing critic for LLM-based data visualization generation. arXiv preprint arXiv:2506.13326 , 2025. URL https://arxiv.org/abs/2506.13326 . - · arXiv
  161. [178][178] W. Pang, K. Q. Lin, X. Jian, X. He, and P. Torr. Paper2Poster: Towards multimodal poster automation from scientific papers. In Advances in Neural Information Processing Systems , 2025. URL https://arxiv.org/abs/ 2505.21497 . -
  162. [179][179] S. S. Panigrahi, J. Videnovic, and M. Brbic. HeurekaBench: A benchmarking framework for AI co-scientist. In International Conference on Learning Representations , 2026. URL https://arxiv.org/abs/2601.01678 . - · arXiv
  163. [180][180] K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. BLEU: A method for automatic evaluation of machine translation. In Annual Meeting on Association for Computational Linguistics , pages 311-318, 2002. -
  164. [181][181] L. Patel, N. Arabzadeh, H. Gupta, A. Sundar, I. Stoica, M. Zaharia, and C. Guestrin. DeepScholar-Bench: A live benchmark and automated evaluation for generative research synthesis. arXiv preprint arXiv:2508.20033 , 2025. URL https://arxiv.org/abs/2508.20033 . - · arXiv
  165. [182][182] P. Pedinotti et al. STRUCTSURVEY: Structured agentic retrieval for automated survey paper generation. arXiv preprint arXiv:2607.01243 , 2026. URL https://arxiv.org/abs/2607.01243 . - · arXiv
  166. [184][184] O. Press, A. Hochlehnert, A. Prabhu, V. Udandarao, O. Press, and M. Bethge. CiteME: Can language models accurately cite scientific claims? arXiv preprint arXiv:2407.12861 , 2024. URL https://arxiv.org/abs/2407.12861 . - · arXiv
  167. [185][185] K. Pu, K. J. K. Feng, T. Grossman, T. Hope, B. D. Mishra, M. Latzke, J. Bragg, J. C. Chang, and P. Siangliulue. IdeaSynth: Iterative research idea development through evolving and composing idea facets with literaturegrounded feedback. In CHI Conference on Human Factors in Computing Systems , pages 1-31, 2025. URL https://arxiv.org/abs/2410.04025 . - · arXiv
  168. [186][186] Y. Pu, T. Lin, and H. Chen. PiFlow: Principle-aware scientific discovery with multi-agent collaboration. arXiv preprint arXiv:2505.15047 , 2025. URL https://arxiv.org/abs/2505.15047 . - · arXiv
  169. [187][187] W. Qian, B. Xu, Z. Xie, B. Fan, G. Tang, J. Chen, X. Wu, M. Yang, C. Di, J. Li, L. Tung, P. Lai, Y. Xia, Z. Guo, Y. Xu, Y. Qin, S. Gan, X. Miao, and B. Cui. AutoSci: A memory-centric agentic system for the full scientific research lifecycle. arXiv preprint arXiv:2605.31468 , 2026. URL https://arxiv.org/abs/2605.31468 . - · arXiv
  170. [188][188] Y. Qiu, H. Zhang, Z. Xu, M. Li, D. Song, Z. Wang, and K. Zhang. AI Idea Bench 2025: AI research idea generation benchmark. arXiv preprint arXiv:2504.14191 , 2025. URL https://arxiv.org/abs/2504.14191 . - · arXiv
  171. [189][189] A. Qu, H. Zheng, Z. Zhou, Y. Yan, Y. Tang, S. Y. Ong, F. Hong, K. Zhou, C. Jiang, M. Kong, J. Zhu, X. Jiang, S. Li, C. Wu, B. K. H. Low, J. Zhao, and P. P. Liang. CORAL: Towards autonomous multi-agent evolution for open-ended discovery. arXiv preprint arXiv:2604.01658 , 2026. URL https://arxiv.org/abs/2604.01658 . - · arXiv
  172. [190][190] Y. Qu, K. Huang, M. Yin, K. Zhan, D. Liu, D. Yin, H. C. Cousins, W. A. Johnson, X. Wang, M. Shah, R. B. Altman, D. Zhou, M. Wang, and L. Cong. CRISPR-GPT for agentic automation of gene-editing experiments. arXiv preprint arXiv:2404.18021 , 2024. URL https://arxiv.org/abs/2404.18021 . - · arXiv
  173. [191][191] V. Raina, A. Liusie, and M. Gales. Is LLM-as-a-judge robust? investigating universal adversarial attacks on zero-shot LLM assessment. In Conference on Empirical Methods in Natural Language Processing , pages 7499-7517, 2024. URL https://arxiv.org/abs/2402.14016 . - · arXiv
  174. [192][192] B. Rank, H. Bhatnagar, A. Prabhu, S. Eisenberg, K. Nguyen, M. Bethge, and M. Andriushchenko. PostTrainBench: Can LLM agents automate LLM post-training? arXiv preprint arXiv:2603.08640 , 2026. URL https://arxiv.org/ abs/2603.08640 . - · arXiv
  175. [193][193] V. Rao, A. Kumar, H. Lakkaraju, and N. B. Shah. Detecting LLM-generated peer reviews. arXiv preprint arXiv:2503.15772 , 2025. URL https://arxiv.org/abs/2503.15772 . - · arXiv
  176. [194][194] M. T. R. Ratul, Z. Chen, K. Fu, T. Ji, and L. Zhang. MasterSet: A large-scale benchmark for must-cite citation recommendation in the AI/ML literature. arXiv preprint arXiv:2604.17680 , 2026. URL https://arxiv.org/abs/ 2604.17680 . - · arXiv
  177. [195][195] J. A. Rodriguez, A. Puri, S. Agarwal, I. H. Laradji, P. Rodriguez, S. Rajeswar, D. Vazquez, C. Pal, and M. Pedersoli. StarVector: Generating scalable vector graphics code from images and text. In IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 16175-16186, 2025. URL https://arxiv.org/abs/2312.11556 . - · arXiv
  178. [196][196] B. Romera-Paredes, M. Barekatain, A. Novikov, M. Balog, M. P. Kumar, E. Dupont, F. J. R. Ruiz, J. S. Ellenberg, P. Wang, O. Fawzi, P. Kohli, and A. Fawzi. Mathematical discoveries from program search with large language models. Nature , 625(7995):468-475, 2024. -
  179. [197][197] K. Ruan, X. Wang, J. Hong, P. Wang, Y. Liu, and H. Sun. LiveIdeaBench: Evaluating LLMs' scientific creativity and idea generation with minimal context. arXiv preprint arXiv:2412.17596 , 2024. URL https: //arxiv.org/abs/2412.17596 . - · arXiv
  180. [198][198] Q. Ruan and I. Gurevych. Author-in-the-loop response generation and evaluation: Integrating author expertise and intent in responses to peer review. arXiv preprint arXiv:2602.11173 , 2026. URL https://arxiv.org/abs/ 2602.11173 . - · arXiv
  181. [199][199] M. Russinovich et al. Phantom references: Hallucinated citations that survive peer review at top-tier conferences. arXiv preprint arXiv:2607.00738 , 2026. URL https://arxiv.org/abs/2607.00738 . - · arXiv
  182. [200][200] G. Russo Latona, M. H. Ribeiro, T. R. Davidson, V. Veselovsky, and R. West. The AI review lottery: Widespread AI-assisted peer reviews boost paper scores and acceptance rates. arXiv preprint arXiv:2405.02150 , 2024. URL https://arxiv.org/abs/2405.02150 . - · arXiv
  183. [201][201] M. Ríos-García, N. Alampara, C. Gupta, I. Mandal, S. Mannan, A. A. Aghajani, N. M. A. Krishnan, and K. M. Jablonka. AI scientists produce results without reasoning scientifically. arXiv preprint arXiv:2604.18805 , 2026. URL https://arxiv.org/abs/2604.18805 . - · arXiv
  184. [202][202] R. Saha, G. Juneja, D. Chaudhuri, N. Sajeevan, N. B. Shah, and D. Pruthi. Policies permitting LLM use for polishing peer reviews are currently not enforceable. arXiv preprint arXiv:2603.20450 , 2026. URL https://arxiv.org/abs/2603.20450 . - · arXiv
  185. [203][203] D. Sahoo, M. Prasad, V. Majhi, J. Singh, V. Chamola, Y. Sinha, M. Mandal, and D. Kumar. When reject turns into accept: Quantifying the vulnerability of LLM-based scientific reviewers to indirect prompt injection. arXiv preprint arXiv:2512.10449 , 2025. URL https://arxiv.org/abs/2512.10449 . - · arXiv
  186. [204][204] G. Sahu, H. Larochelle, L. Charlin, and C. Pal. ReviewerToo: Should AI join the program committee? a look at the future of peer review. arXiv preprint arXiv:2510.08867 , 2025. URL https://arxiv.org/abs/2510.08867 . - · arXiv
  187. [205][205] A. Sanyal, S. Schapiro, S. Shashidhar, R. Moon, L. R. Varshney, and D. Hakkani-Tur. Spark: A system for scientifically creative idea generation. In International Conference on Computational Creativity , 2025. URL https://arxiv.org/abs/2504.20090 . - · arXiv
  188. [206][206] T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, L. Zettlemoyer, N. Cancedda, and T. Scialom. Toolformer: Language models can teach themselves to use tools. In Advances in Neural Information Processing Systems , volume 36, pages 68539-68551, 2023. URL https://arxiv.org/abs/2302.04761 . - · arXiv
  189. [207][207] S. Schmidgall and M. Moor. AgentRxiv: Towards collaborative autonomous research. arXiv preprint arXiv:2503.18102 , 2025. URL https://arxiv.org/abs/2503.18102 . - · arXiv
  190. [208][208] S. Schmidgall, Y. Su, Z. Wang, X. Sun, J. Wu, X. Yu, J. Liu, M. Moor, Z. Liu, and E. Barsoum. Agent laboratory: Using LLM agents as research assistants. In Conference on Empirical Methods in Natural Language Processing , pages 5977-6043, 2025. URL https://arxiv.org/abs/2501.04227 . - · arXiv
  191. [209][209] N. Schuster, A. N. Salcedo, S. Bouchard, D. Frei, A. Pisani, J. E. Bautista, J. Zoubian, S. Escoffier, W. Liu, G. Valogiannis, and P. Zarrouk. Setting SAIL: Leveraging scientist-AI-loops for rigorous visualization tools. arXiv preprint arXiv:2603.18145 , 2026. URL https://arxiv.org/abs/2603.18145 . - · arXiv
  192. [211][211] M. Seo, J. Baek, S. Lee, and S. J. Hwang. Paper2Code: Automating code generation from scientific papers in machine learning. arXiv preprint arXiv:2504.17192 , 2025. URL https://arxiv.org/abs/2504.17192 . - · arXiv
  193. [212][212] F. Shahhosseini, A. Marioriyad, A. Momen, M. S. Baghshah, M. H. Rohban, and S. H. Javanmard. Large language models for scientific idea generation: A creativity-centered survey. arXiv preprint arXiv:2511.07448 , 2025. URL https://arxiv.org/abs/2511.07448 . - · arXiv
  194. [213][213] C. Shao, Y. Li, and F. Xu. SciNetBench: A relation-aware benchmark for scientific literature retrieval agents. arXiv preprint arXiv:2601.03260 , 2026. URL https://arxiv.org/abs/2601.03260 . - · arXiv
  195. [214][214] E. Shao, Y. Wang, Y. Qian, Z. Pan, H. Liu, and D. Wang. SciSciGPT: Advancing human-AI collaboration in the science of science. Nature Computational Science , 6:301-315, 2026. URL https://arxiv.org/abs/2504.05559 . - · arXiv
  196. [215][215] Y. Shao, Y. Jiang, T. A. Kanell, P. Xu, O. Khattab, and M. S. Lam. Assisting in writing Wikipedialike articles from scratch with large language models. arXiv preprint arXiv:2402.14207 , 2024. URL https: //arxiv.org/abs/2402.14207 . - · arXiv
  197. [216][216] V. Sharma, T. Joachims, and S. Dean. Do LLMs favor LLMs? quantifying interaction effects in peer review. arXiv preprint arXiv:2601.20920 , 2026. URL https://arxiv.org/abs/2601.20920 . - · arXiv
  198. [217][217] H. Shen, H. Yang, Z. Gu, and W. Han. ScholarGym: Benchmarking large language model capabilities in the information-gathering stage of deep research. arXiv preprint arXiv:2601.21654 , 2026. URL https: //arxiv.org/abs/2601.21654 . - · arXiv
  199. [218][218] C. Shi, Q. Cai, Z. Chen, L. Zeng, Y. Zhao, J. Yu, J. Yu, and X. Li. APEX: Academic poster editing agentic expert. arXiv preprint arXiv:2601.04794 , 2026. URL https://arxiv.org/abs/2601.04794 . - · arXiv
  200. [219][219] J. Shi, Z. Zhang, B. Wu, Y. Liang, M. Fang, L. Chen, and Y. Zhao. PresentAgent: Multimodal agent for presentation video generation. In Conference on Empirical Methods in Natural Language Processing , pages 760-773, 2025. URL https://arxiv.org/abs/2507.04036 . - · arXiv

200 of 312 shown

Cited by

Sorted by citations

Authors
Kong, L., Sun, X., Chow, W., Li, L., Lin, K. Q., Zhang, X. B., Wang, S., Li, R., Wu, Q., Gao, W., Wang, Y., Xie, S., Liu, J., Qu, L., Li, S., Ng, L. X., Cottereau, B. R., Liu, Z., Chua, T.-S., & Ooi, W. T.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2605.18661
arXiv
2605.18661
OpenAlex
W7161721808
Cited by
0
References
0
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
Keywords
AI-assisted research, large language models, research integrity, peer review, manuscript revision, research lifecycle, human-AI collaboration, AI failure modes, research dissemination, benchmark suite

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review