The DOI matches its DataCite record; 6 of the 6 references it cites were found in a registry or an index.
Agent Laboratory: Using LLM Agents as Research Assistants
Abstract
Historically, scientific discovery has been a lengthy and costly process, demanding substantial time and resources from initial conception to final results. To accelerate scientific discovery, reduce research costs, and improve research quality, we introduce Agent Laboratory, an autonomous LLM-based framework capable of completing the entire research process. This framework accepts a human-provided research idea and progresses through three stages--literature review, experimentation, and report writing to produce comprehensive research outputs, including a code repository and a research report, while enabling users to provide feedback and guidance at each stage. We deploy Agent Laboratory with various state-of-the-art LLMs and invite multiple researchers to assess its quality by participating in a survey, providing human feedback to guide the research process, and then evaluate the final paper. We found that: (1) Agent Laboratory driven by o1-preview generates the best research outcomes; (2) The generated machine learning code is able to achieve state-of-the-art performance compared to existing methods; (3) Human involvement, providing feedback at each stage, significantly improves the overall quality of research; (4) Agent Laboratory significantly reduces research expenses, achieving an 84% decrease compared to previous autonomous research methods. We hope Agent Laboratory enables researchers to allocate more effort toward creative ideation rather than low-level coding and writing, ultimately accelerating scientific discovery.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It fills a gap in the current research landscape where there are many such agents, e.g., AI Scientist (Lu et al., 2024), AgentLab (Schmidgall et al., 2025), and CodeScientist (Jansen et al., 2025), but no systematic way to compare them.”
From AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite (2025) · cites this paper as Schmidgall et al., 2025
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Self-citationLLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Ashkinaze, J., Mendelsohn, J., Qiwei, L., Budak, C., & Gilbert, E. (2025) · Proceedings of the ACM Collective Intelligence Conference · doi:10.1145/3715928.3737481 · Cited by 33
Dohmke, T., Iansiti, M., & Richards, G. (2023) · arXiv · doi:10.48550/arxiv.2306.15033 · Cited by 10
On the Impacts of Contexts on Repository-Level Code Generation
Hai, N. L., Nguyen, D. M., & Bui, N. D. Q. (2024) · arXiv · doi:10.48550/arxiv.2406.11927 · Cited by 3
DSBench: How Far Are Data Science Agents from Becoming Data Science Experts?
Jing, L., Huang, Z., Wang, X., Yao, W., Yu, W., Ma, K., Zhang, H., Du, X., & Yu, D. (2024) · arXiv · doi:10.48550/arxiv.2409.07703 · Cited by 1
Cited by
Large Language Models for Information Retrieval: A Survey
Zhu, Y., Yuan, H., Wang, S., Liu, J., Liu, W., Deng, C., Chen, H., Liu, Z., Dou, Z., & Wen, J.-R. (2026) · ACM Transactions on Information Systems · doi:10.1145/3748304 · Cited by 225
Self-citationAgentic Large Language Models, a Survey
Plaat, A., Van Duijn, M., Van Stein, N., Preuss, M., Van der Putten, P., & Batenburg, K. J. (2025) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.18675 · Cited by 75
AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite
Bragg, J., D'Arcy, M., Balepur, N., Bareket, D., Dalvi, B., Feldman, S., Haddad, D., Hwang, J. D., Jansen, P., Kishore, V., et al. (2025) · arXiv · doi:10.48550/arxiv.2510.21652 · Cited by 63
The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search
Yamada, Y., Lange, R. T., Lu, C., Hu, S., Lu, C., Foerster, J., Clune, J., & Ha, D. (2025) · arXiv · doi:10.48550/arxiv.2504.08066 · Cited by 32
GridMind: LLMs-Powered Agents for Power System Analysis and Operations
Jin, H., Kim, K., & Kwon, J. (2025) · Proceedings of the SC '25 Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis · doi:10.1145/3731599.3767409 · Cited by 20
Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directions
Gridach, M., Nanavati, J., Abidine, K. Z. E., Mendes, L., & Mack, C. (2025) · arXiv · doi:10.48550/arxiv.2503.08979 · Cited by 18
ToPolyAgent: AI agents for coarse-grained bead-spring topological polymer simulations
Ding, L., Carrillo, J.-M., & Do, C. (2026) · Digital Discovery · doi:10.1039/d5dd00471c · Cited by 11
“The application of AI agent started with software development[47, 48], and has been quickly expanding to various areas of scientific research[49–51].”
Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents
de Witt, C. S., Krawiecka, K., Krawczuk, I., Hagag, B., Anderson, W. L., Belcak, P., Bucknall, B., Cai, X., Chopra, A., Cohen, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.02077 · Cited by 108
“…multi-step tasks online (Lù et al., 2024; Putta et al., 2024) - booking travel arrangements, conducting in-depth research (Gottweis et al., 2025; Schmidgall et al., 2025), and negotiating transactions, or using computers (Humphreys et al., 2022; Bonatti et al., 2024) through interfaces…”
SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Papers
Xiang, Y., Yan, H., Ouyang, S., Gui, L., & He, Y. (2025) · arXiv · doi:10.48550/arxiv.2504.00255 · Cited by 1
A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agents
Huang, W.-C., Zhang, W., Liang, Y., Bei, Y., Chen, Y., Feng, T., Pan, X., Tan, Z., Wang, Y., Wei, T., et al. (2026) · arXiv · doi:10.48550/arxiv.2602.06052 · Cited by 70
ReproAgent: Contract-Guided Paper-to-Code Reproduction
Hu, X., Pan, Z., Wang, Z., Liu, Z., Su, Z., & Zhang, W. (2026) · arXiv · doi:10.48550/arxiv.2608.24291
Self-citationShi, X., Tian, Z., Tan, S., & Wang, X. (2026) · medRxiv · doi:10.64898/2026.04.02.26350091
Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning
Seo, M., Baek, J., Lee, S., & Hwang, S. J. (2025) · arXiv · doi:10.48550/arxiv.2504.17192
DeepScientist: Advancing Frontier-Pushing Scientific Findings Progressively
Weng, Y., Zhu, M., Xie, Q., Sun, Q., Lin, Z., Liu, S., & Zhang, Y. (2025) · arXiv · doi:10.48550/arxiv.2509.26603
Accelerating scientific discovery with Co-Scientist
Gottweis, J., Weng, W.-H., Daryin, A., Tu, T., Sirkovic, P., Myaskovsky, A., Glowaty, G., Weissenberger, F., Orlandi, A., Popovici, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2502.18864 · Cited by 158
Metadata
- Authors
- Schmidgall, S., Su, Y., Wang, Z., Sun, X., Wu, J., Yu, X., Liu, J., Moor, M., Liu, Z., & Barsoum, E.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- arXiv
- 2501.04227
- References
- 6
- Type
- Preprint
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
