The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index; 3 of the 16 sources its claims cite were checked for support.
Towards end-to-end automation of AI research
Open accessAbstract
The automation of science is a long-standing ambition in artificial intelligence (AI) research 1,2 . Although the community has made substantial progress in automating individual components of the scientific process, a system that autonomously navigates the entire research life cycle—from conception to publication—has remained out of reach. Here we present a pipeline for automating the entire scientific process end to end. We present The AI Scientist, which creates research ideas, writes code, runs experiments, plots and analyses data, writes the entire scientific manuscript, and performs its own peer review. Its ideas, execution and presentation are of sufficient quality that the manuscript generated by this AI system passed the first round of peer review for a workshop of a top-tier machine learning conference. The workshop had an acceptance rate of 70%. Our system leverages modern foundation models 3–5 within a complex agentic system. We evaluate The AI Scientist in two settings: a focused mode using human-provided code templates as an initial scaffold for conducting research on a specific topic and a template-free, open-ended mode that leverages agentic search for wider scientific exploration 6,7 . Both settings produce diverse ideas and automatically test, report on and evaluate them. This achievement demonstrates the growing capacity of AI for making scientific contributions and signifies a potential paradigm shift in how research is conducted. As with any impactful new technology, there could be important risks, including taxing overwhelmed review systems and adding noise to the scientific literature. However, if developed responsibly, such autonomous systems could greatly accelerate scientific discovery.
Overview
CoChat Summary
TL;DR
The paper presents The AI Scientist, an agentic pipeline that autonomously generates machine-learning research ideas, code, experiments, plots, manuscripts, and peer reviews. One fully AI-generated manuscript received an average score of 6.33 (6, 7, 6) at an ICLR 2025 workshop, above the workshop’s average acceptance threshold, though it was withdrawn per protocol.
Key findings
- The AI Scientist completed an end-to-end research workflow; one of three submissions to the ICLR 2025 ICBINB workshop received an average score of 6.33 (individual scores 6, 7, 6), above the average acceptance threshold. The organizers said it would likely have been accepted if not withdrawn; the workshop acceptance rate was 70% versus 32% for the ICLR 2025 main conference.
- The Automated Reviewer performed comparably to human reviewers: balanced accuracy 69% versus 66%, and F1 score 0.62 versus 0.49 for inter-human agreement. On contamination checks, balanced accuracy decreased from 69% before the knowledge cutoff to 66% in the clean 2025 dataset, remaining comparable to human reviewers.
- Paper quality increased with better base models and more test-time compute; recent model generations produced papers approaching borderline acceptability for machine-learning conference workshops.
- The template-free system can generate its own starting code, uses agentic tree search, and typically takes from several hours to over 15 h depending on problem complexity.
Methods
- Two AI Scientist modes: template-based (human-provided code template, Aider, sequential plan with up to five experiments, 7,200 s timeout per experiment, up to four debug reattempts) and template-free (open-ended, parallelized agentic tree search, four experiment stages, VLM critique, HuggingFace dataset access, direct LaTeX generation). It uses foundation models including o3, Claude Sonnet 4, GPT-4o, o4-mini, and o1.
- The Automated Reviewer uses o4-mini, follows NeurIPS reviewer guidelines, produces structured scores and accept/reject decisions, and ensembles five reviews plus a meta-review acting as area chair. It was validated on ICLR OpenReview data, including 1,000 papers from 2017–2024 and a clean 2025 dataset.
- Human evaluation submitted three generated manuscripts to the ICLR 2025 ICBINB workshop with IRB approval and organizer cooperation; reviewers were blind to which submissions were AI-generated, and all AI-generated submissions were withdrawn after review regardless of outcome.
Limitations
- Stated: only one of three submissions was accepted, and workshops have much higher acceptance rates than main conferences (70% versus 32%). The AI Scientist cannot yet meet top-tier publication standards nor consistently meet workshop standards; failure modes include naive or underdeveloped ideas, incorrect implementations, lack of deep methodological rigour, experimental errors, duplicated figures, and hallucinations such as inaccurate citations.
- Stated: it currently conducts computational experiments only. Ethical and societal concerns include overwhelming peer review, artificially inflating research credentials, repurposing others’ ideas without credit, eliminating scientist jobs, and conducting unethical or dangerous experiments; more research is needed for safe, aligned open-ended exploratory AI.
- Evident from the text: human filtering selected the most promising outputs at each stage before submission, so the submitted set was not fully unfiltered autonomous output; the paper notes that without filtering, final papers would still be produced but along with other papers at greater total cost. The paper also states that the comparison between Automated Reviewer and human reviewers is not exact because ICLR and NeurIPS paper pools differ, creating a distribution shift.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
2 Valid · 2 Not valid · 9 Not checked · 50 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, The Automated Reviewer shows that improvements to the base models significantly improve the quality of the generated papers”
§ Towards end-to-end automation of AI research > Main
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“One of the three AI-generated manuscripts received an average score of 6.33 from the reviewers (individual scores were 6, 7 and 6), placing it above the average acceptance threshold for the workshop (Fig. 2).”
§ Towards end-to-end automation of AI research > Human evaluation results
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Its ideas, execution and presentation are of sufficient quality that the manuscript generated by this AI system passed the first round of peer review for a workshop of a top-tier machine learning conference.”
- Also stated in: § Main · plain
“One of The AI Scientist’s manuscripts achieved high enough scores to exceed the average human acceptance threshold at a workshop, providing an example of a fully AI-generated paper successfully navigating a peer-review process, albeit one with a lower bar.”
§ Towards end-to-end automation of AI research > Abstract
plainSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Main · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The other two papers did not meet the bar for acceptance (Supplementary Table D9).”
§ Towards end-to-end automation of AI research > Human evaluation results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“These results are statistically significant (non-parametric bootstrap test35 and two-sample z-test36; Supplementary Information section A.3).”
§ Towards end-to-end automation of AI research > Automated evaluation of generated papers
What the sources say
- Not checked — no full text of the sourcecites [35] Efron, B. & Tibshirani, R. J. An Introduction to the Bootstrap (Chapman & Hall CRC, 1993). source
- Not matchedcites [36] Lehmann, E. L. Testing Statistical Hypotheses 1st edn (Wiley, 1959).
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
23 more claims
Create a free account to see all 63 claims and every source verdict.
- It may state two separate claims as one. 20
- Probably not a substantial claim. 24
- The quote only partly states this claim. 1
Citations
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
Deep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 4,688 · cited 1×
“However, some AI weaknesses have proved surprisingly difficult to solve, such as AI being easily fooled 41,42 and overconfidently wrong (hallucinations) 43 , although progress has been made 44,45 .”
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
Nguyen, A., Yosinski, J., & Clune, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298640 · Cited by 3,258 · cited 1×
“However, some AI weaknesses have proved surprisingly difficult to solve, such as AI being easily fooled 41,42 and overconfidently wrong (hallucinations) 43 , although progress has been made 44,45 .”
Self-citationOn the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · cited 1×
“Finally, more research is needed to ensure that open-ended exploratory AI proceeds safely and in alignment with human values 47,48 .”
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941 · cited 1×
“Agentic patterns 49 , such as few-shot prompting 50 and self-reflection 51 , are leveraged by The AI Scientist to improve performance and reliability.”
Research works printed but not linked: 16
- Lenat, D. B. Automated theory formation in mathematics. In Proc. 5th International Joint Conference on Artificial Intelligence 833–842 (ed. Reddy, R.) (William Kaufmann, 1977).
- The Claude 3 Model Family: Opus, Sonnet, Haiku https://api.semanticscholar.org/CorpusID:268232499 (Anthropic, 2024).
- Chan, J. S. et al. MLE-bench: evaluating machine learning agents on machine learning engineering. In Proc. 13th International Conference on Learning Representation (ed. Yue, Y. et al.) (ICLR, 2025).
- Faldor, M., Zhang, J., Cully, A. & Clune, J. OMNI-EPIC: open-endedness via models of human notions of interestingness with environments programmed in code. In Proc. 13th International Conference on Learning Representations (eds Yue, Y. et al.) (ICLR, 2025).
- Hu, S., Lu, C. & Clune, J. Automated design of agentic systems. In Proc. 13th International Conference on Learning Representations (eds Yue, Y. et al.) (ICLR, 2025).
- Huang, Q., Vora, J., Liang, P. & Leskovec, J. MLAgentBench: evaluating language agents on machine learning experimentation. In Proc. 41st International Conference on Machine Learning Vol. 235 (eds Salakhutdinov, R. et al.) 20271–20309 (PMLR, 2024).
- Ma, Y. J. et al. Eureka: human-level reward design via coding large language models. In Proc. 12th International Conference on Learning Representations (eds Kim, B. et al.) (ICLR, 2024).
- González-Márquez, R. & Kobak, D. Learning representations of learning representations. In Proc. Data-centric Machine Learning Research (DMLR) Workshop at ICLR (ICLR, 2024).
- Beygelzimer, A., Dauphin, Y., Liang, P. & Vaughan, J. W. The NeurIPS 2021 consistency experiment. NeurIPS Blog https://blog.neurips.cc/2021/12/08/the-neurips-2021-consistency-experiment (2021).
- Lehmann, E. L. Testing Statistical Hypotheses 1st edn (Wiley, 1959).
- I Can’t Believe it’s Not Better: Challenges in Applied Deep Learning. ICLR Workshop at ICLR 2025 http://openreview.net/group?id=ICLR.cc/2025/Workshop/ICBINB (OpenReview, 2025).
- 13th Annual International Conference on Learning Representations (ICLR) 2025 Fact Sheet https://media.iclr.cc/Conferences/ICLR2025/ICLR2025_Fact_Sheet.pdf (ICLR, 2025).
- Szegedy, C. et al. Intriguing properties of neural networks. In Proc. 2nd International Conference on Learning Representations (eds Bengio, Y. & LeCun, Y.) (ICLR, 2014).
- Brown, T. et al. Language models are few-shot learners. In Proc. Advances in Neural Information Processing Systems Vol. 33 (eds Larochelle, H. et al.) 1877–1901 (Curran Associates, 2020).
- Vaswani, A. et al. Attention is all you need. In Advances in Neural Information Processing Systems (eds Guyon, I. et al.) Vol. 30, 5998–6008 (Curran Associates, 2017).
- El-Kishky, A. OpenAI o1 System Card https://api.semanticscholar.org/CorpusID:272648256 (OpenAI, 2024).
Cited by
Assessing the (In)Ability of LLMs to Reason in Interval Temporal Logic
Bellodi, P., Casavecchia, P., Paparella, A., Sciavicco, G., & Stan, I. E. (2025) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.time.2025.4 · Cited by 1
A multi-agent system for automating scientific discovery
Ghareeb, A. E., Chang, B., Mitchener, L., Angela, Y., Szostkiewicz, C. J., Shved, D., Gyimesi, G. J., Laurent, J. M., Wright, S. M., Razzak, M. T., et al. (2026) · Nature · doi:10.1038/s41586-026-10652-y · Cited by 452
Kim, S., Yoon, D., Gashteovski, K., Suk, J., Baek, J., Aggarwal, P., Wu, I., Zaverkin, V., Petkoski, S., Schrider, D. R., et al. (2026) · arXiv · doi:10.48550/arxiv.2605.20668 · Cited by 15
Why AI cannot do good science without humans
(2026) · Nature · doi:10.1038/d41586-026-01551-3 · Cited by 10
The uncritical adoption of AI in science is alarming — we urgently need guard rails
Messeri, L., & Crockett, M. J. (2026) · Nature · doi:10.1038/d41586-026-01557-x · Cited by 7
Metadata
- Authors
- Lu, C., Lu, C., Lange, R. T., Yamada, Y., Hu, S., Foerster, J., Ha, D., & Clune, J.
- Affiliations
- University of British Columbia (CA)
- Sakana AI (Japan) (JP)
- University of Oxford (GB)
- Corresponding author
- Yamada, Yutaro
- Ha, David
- Clune, Jeff
- Venue
- Nature
- Published
- 2026-03-25
- Volume
- 651
- Issue
- 8107
- Pages
- 914-919
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- PMC
- PMC13017497
- OpenAlex
- W7140287209
- Semantic Scholar
- b73e2b7ffbb00b03b795dc8b5e732a8859470249
- Cited by
- 116
- References
- 41
- Citations by year
- 2026: 115
- FWCI
- 387.07
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Nature (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Canadian Institute for Advanced Research
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Keywords
- research idea generation, code generation, manuscript preparation, peer review, foundation models, agentic systems, agentic search, research lifecycle, machine learning conferences
- MeSH terms
- Machine Learning, Artificial Intelligence, Automation, Humans, Research, Research Personnel, Peer Review, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Towards end-to-end automation of AI research. https://app.cochat.ai/papers/10.1038/s41586-026-10265-5
- The paper
- Lu, C., Lu, C., Lange, R. T., Yamada, Y., Hu, S., Foerster, J., Ha, D., & Clune, J. (2026). Towards end-to-end automation of AI research. Nature. https://doi.org/10.1038/s41586-026-10265-5
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.