Autonomous LLM-Driven Research — from Data to Human-Verifiable Research Papers
Open accessThe DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Artificial intelligence (AI) promises to accelerate scientific discovery, but it remains unclear whether AI systems can perform fully autonomous research, and whether they can do so while adhering to key scientific values, such as transparency, traceability, and verifiability. The aim of this study was to develop and evaluate an AI-automation platform that performs transparent, traceable, and human-verifiable scientific research. METHODS: To mimic human scientific practices, we built "data-to-paper," an automation platform that guides interacting large language model (LLM) agents through a complete stepwise research process that starts with annotated data and results in comprehensive research papers, while programmatically backtracing information flow and allowing human oversight and interactions. The platform can run fully autonomously (in autopilot mode) or with human intervention (in copilot mode). RESULTS: In autopilot mode, provided only with annotated data, data-to-paper raised hypotheses; designed research plans; wrote and debugged analysis codes; generated and interpreted results; and created complete, information-traceable research papers. Even though the research novelty of manuscripts created by data-to-paper was relatively limited, the process demonstrated the autonomous generation of de novo quantitative insights from data, such as unraveling associations between health indicators and clinical outcomes. For simple research goals and datasets, a fully autonomous cycle can create manuscripts that independently recapitulate the findings of peer-reviewed biomedical publications without major errors in about 80 to 90% of cases. Yet, as goal or data complexity increases, human copiloting becomes critical for ensuring accuracy and overall quality. By tracking information flow through the steps, the platform creates "data-chained" manuscripts, in which downstream results are programmatically linked to upstream code and data, thus setting a new standard for the verifiability of scientific outputs. CONCLUSIONS: Our work demonstrates the potential for AI-driven acceleration of scientific discovery in data-driven biomedical research and beyond, while enhancing, rather than jeopardizing, traceability, transparency, and verifiability.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
A Survey on Large Language Model based Autonomous Agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.11432 · Cited by 1,925
Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He, H., Li, A., He, M., Liu, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.01852 · Cited by 1,172
SPECTER: Document-level Representation Learning using Citation-informed Transformers
Cohan, A., Feldman, S., Beltagy, I., Downey, D., & Weld, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.207 · Cited by 1,108
Autonomous chemical research with large language models
Boiko, D. A., MacKnight, R., Kline, B., & Gomes, G. (2023) · Nature · doi:10.1038/s41586-023-06792-0 · Cited by 1,050
Competition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043
Competition-Level Code Generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Lago, A. D., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.07814 · Cited by 1,043
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970
ChatGPT listed as author on research papers: many scientists disapprove
Stokel-Walker, C. (2023) · Nature · doi:10.1038/d41586-023-00107-z · Cited by 868
Summary of ChatGPT-Related research and perspective towards the future of large language models
Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He, H., Li, A., He, M., Liu, Z., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100017 · Cited by 855
Research works printed but not linked: 14
- Tu T Palepu A Schaekermann M et al. Towards conversational diagnostic AI. January 11 2024 (http://arXiv.org/abs/2401.05654). Preprint. · arXiv
- 10.1056/AIoa2400196 · doi:10.1056/aioa2400196 · Paper page
- Wilson EB. An introduction to scientific research. McGraw-Hill Book Company, 1952.
- Ji Z Lee N Frieske R et al. Survey of hallucination in natural language generation. February 8 2022 (http://arXiv.org/abs/2202.03629). Preprint. · arXiv
- Touvron H Martin L Stone K et al. Llama 2: open foundation and fine-tuned chat models. July 18 2023 (http://arXiv.org/abs/2307.09288). Preprint. · arXiv
- Rozière B Gehring J Gloeckle F et al. Code Llama: open foundation models for code. August 24 2023 (http://arXiv.org/abs/2308.12950). Preprint. · arXiv
- Wu Q Bansal G Zhang J et al. AutoGen: enabling next-gen LLM applications via multi-agent conversation. August 16 2023 (http://arXiv.org/abs/2308.08155). Preprint. · arXiv
- Hong S Zhuge M Chen J et al. MetaGPT: meta programming for a multi-agent collaborative framework. August 1 2023 (http://arXiv.org/abs/2308.00352). Preprint. · arXiv
- Madaan A Tandon N Gupta P et al. Self-refine: iterative refinement with self-feedback. March 30 2023 (http://arXiv.org/abs/2303.17651). Preprint. · arXiv
- Rolle-Lake L, Robbins E. Behavioral risk factor surveillance system. In: StatPearls. Treasure Island, FL: StatPearls Publishing, 2023 (https://www.ncbi.nlm.nih.gov/pubmed/31971707).
- Majumder BP Surana H Agarwal D et al. Data-driven discovery with large generative models. February 21 2024 (http://arXiv.org/abs/2402.13610). Preprint. · arXiv
- Abdel-Rehim A Zenil H Orhobor O et al. Scientific hypothesis generation by a large language model: laboratory validation in breast cancer treatment. May 20 2024 (http://arXiv.org/abs/2405.12258). Preprint. · arXiv
- Wang Q Downey D Ji H Hope T. SciMON: scientific inspiration machines optimized for novelty. May 23 2023 (http://arXiv.org/abs/2305.14259). Preprint. · arXiv
- Lu C Lu C Lange RT Foerster J Clune J Ha D. The AI scientist: towards fully automated open-ended scientific discovery. August 12 2024 (http://arXiv.org/abs/2408.06292). Preprint. · arXiv
Cited by
Artificial Intelligence for Materials Discovery, Development, and Optimization
Madika, B., Saha, A., Kang, C., Buyantogtokh, B., Agar, J., Wolverton, C. M., Voorhees, P., Littlewood, P., Kalinin, S., & Hong, S. (2025) · ACS Nano · doi:10.1021/acsnano.5c04200 · Cited by 171
Towards end-to-end automation of AI research
Lu, C., Lu, C., Lange, R. T., Yamada, Y., Hu, S., Foerster, J., Ha, D., & Clune, J. (2026) · Nature · doi:10.1038/s41586-026-10265-5 · Cited by 116
A multi-agent system for automating scientific discovery
Ghareeb, A. E., Chang, B., Mitchener, L., Angela, Y., Szostkiewicz, C. J., Shved, D., Gyimesi, G. J., Laurent, J. M., Wright, S. M., Razzak, M. T., et al. (2026) · Nature · doi:10.1038/s41586-026-10652-y · Cited by 454
How can artificial intelligence transform the training of medical students and physicians?
Ning, Y., Ong, J. C. L., Cheng, H., Wang, H., Ting, D. S. W., Tham, Y. C., Wong, T. Y., & Liu, N. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.100900 · Cited by 77
Responsible AI in biotechnology: balancing discovery, innovation and biosecurity risks
Wheeler, N. E. (2025) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2025.1537471 · Cited by 71
Metadata
- Authors
- Ifargan, T., Hafner, L., Kern, M., Alcalay, O., & Kishony, R.
- Affiliations
- Technion – Israel Institute of Technology (IL)
- Epsio, Tel Aviv, Israel
- Venue
- NEJM AI
- Publisher
- Massachusetts Medical Society
- Published
- 2025-01-01
- Volume
- 2
- Issue
- 1
- Pages
- AIoa2400555-AIoa2400555
- ISSN
- 2836-9386 (ISSN-L)
- OpenAlex
- W4404941741
- Cited by
- 34
- References
- 26
- Citations by year
- 2026: 13 · 2025: 20
- FWCI
- 50.38
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- NEJM AI (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- large language models, autonomous research, human verification, data-driven research, AI-assisted scientific writing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Autonomous LLM-Driven Research — from Data to Human-Verifiable Research Papers. https://app.cochat.ai/papers/10.1056/aioa2400555
- The paper
- Ifargan, T., Hafner, L., Kern, M., Alcalay, O., & Kishony, R. (2025). Autonomous LLM-Driven Research — from Data to Human-Verifiable Research Papers. NEJM AI. https://doi.org/10.1056/aioa2400555
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.