Autonomous LLM-Driven Research — from Data to Human-Verifiable Research Papers
Open accessThe DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Artificial intelligence (AI) promises to accelerate scientific discovery, but it remains unclear whether AI systems can perform fully autonomous research, and whether they can do so while adhering to key scientific values, such as transparency, traceability, and verifiability. The aim of this study was to develop and evaluate an AI-automation platform that performs transparent, traceable, and human-verifiable scientific research. METHODS: To mimic human scientific practices, we built "data-to-paper," an automation platform that guides interacting large language model (LLM) agents through a complete stepwise research process that starts with annotated data and results in comprehensive research papers, while programmatically backtracing information flow and allowing human oversight and interactions. The platform can run fully autonomously (in autopilot mode) or with human intervention (in copilot mode). RESULTS: In autopilot mode, provided only with annotated data, data-to-paper raised hypotheses; designed research plans; wrote and debugged analysis codes; generated and interpreted results; and created complete, information-traceable research papers. Even though the research novelty of manuscripts created by data-to-paper was relatively limited, the process demonstrated the autonomous generation of de novo quantitative insights from data, such as unraveling associations between health indicators and clinical outcomes. For simple research goals and datasets, a fully autonomous cycle can create manuscripts that independently recapitulate the findings of peer-reviewed biomedical publications without major errors in about 80 to 90% of cases. Yet, as goal or data complexity increases, human copiloting becomes critical for ensuring accuracy and overall quality. By tracking information flow through the steps, the platform creates "data-chained" manuscripts, in which downstream results are programmatically linked to upstream code and data, thus setting a new standard for the verifiability of scientific outputs. CONCLUSIONS: Our work demonstrates the potential for AI-driven acceleration of scientific discovery in data-driven biomedical research and beyond, while enhancing, rather than jeopardizing, traceability, transparency, and verifiability.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
A Survey on Large Language Model based Autonomous Agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.11432 · Cited by 1,925
Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He, H., Li, A., He, M., Liu, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.01852 · Cited by 1,172
SPECTER: Document-level Representation Learning using Citation-informed Transformers
Cohan, A., Feldman, S., Beltagy, I., Downey, D., & Weld, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.207 · Cited by 1,108
Autonomous chemical research with large language models
Boiko, D. A., MacKnight, R., Kline, B., & Gomes, G. (2023) · Nature · doi:10.1038/s41586-023-06792-0 · Cited by 1,050
Competition-level code generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022) · Science · doi:10.1126/science.abq1158 · Cited by 1,043
Competition-Level Code Generation with AlphaCode
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Lago, A. D., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.07814 · Cited by 1,043
The Rise and Potential of Large Language Model Based Agents: A Survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07864 · Cited by 972
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970
ChatGPT listed as author on research papers: many scientists disapprove
Stokel-Walker, C. (2023) · Nature · doi:10.1038/d41586-023-00107-z · Cited by 868
Summary of ChatGPT-Related research and perspective towards the future of large language models
Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He, H., Li, A., He, M., Liu, Z., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100017 · Cited by 855
What ChatGPT and generative AI mean for science
Stokel-Walker, C., & Van Noorden, R. (2023) · Nature · doi:10.1038/d41586-023-00340-6 · Cited by 828
Artificial intelligence and illusions of understanding in scientific research
Messeri, L., & Crockett, M. J. (2024) · Nature · doi:10.1038/s41586-024-07146-0 · Cited by 707
Liang, W., Zhang, Y., Cao, H., Wang, B., Ding, D., Yang, X., Vodrahalli, K., He, S., Smith, D., Yin, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.01783 · Cited by 633
Mathematical discoveries from program search with large language models
Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J. R., Ellenberg, J. S., Wang, P., Fawzi, O., et al. (2024) · Nature · doi:10.1038/s41586-023-06924-6 · Cited by 544
Science in the age of large language models
Birhane, A., Kasirzadeh, A., Leslie, D., & Wachter, S. (2023) · Nature Reviews Physics · doi:10.1038/s42254-023-00581-4 · Cited by 344
Solving olympiad geometry without human demonstrations
Trinh, T. H., Wu, Y., Le, Q. V., He, H., & Luong, T. (2024) · Nature · doi:10.1038/s41586-023-06747-5 · Cited by 311
Self-collaboration Code Generation via ChatGPT
Dong, Y., Jiang, X., Jin, Z., & Li, G. (2023) · arXiv · doi:10.48550/arxiv.2304.07590 · Cited by 256
AI model GPT-3 (dis)informs us better than humans
Spitale, G., Biller-Andorno, N., & Germani, F. (2023) · Science Advances · doi:10.1126/sciadv.adh1850 · Cited by 241
Could AI help you to write your next paper?
Hutson, M. (2022) · Nature · doi:10.1038/d41586-022-03479-w · Cited by 236
Peng, B., Galley, M., He, P., Cheng, H., Xie, Y., Hu, Y., Huang, Q., Liden, L., Yu, Z., Chen, W., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.12813 · Cited by 156
P values and the search for significance
Altman, N., & Krzywinski, M. I. (2016) · Nature Methods · doi:10.1038/nmeth.4120 · Cited by 145
How ChatGPT and other AI tools could disrupt scientific publishing
Conroy, G. (2023) · Nature · doi:10.1038/d41586-023-03144-w · Cited by 121
Prevalence of nonsensical algorithmically generated papers in the scientific literature
Cabanac, G., & Labbé, C. (2021) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24495 · Cited by 83
AI tools can improve equity in science
Berdejo-Espinola, V., & Amano, T. (2023) · Science · doi:10.1126/science.adg9714 · Cited by 83
Living guidelines for generative AI — why scientists must oversee its use
Bockting, C. L., van Dis, E. A. M., van Rooij, R., Zuidema, W., & Bollen, J. (2023) · Nature · doi:10.1038/d41586-023-03266-1 · Cited by 82
ChatDev: Communicative Agents for Software Development
Qian, C., Liu, W., Liu, H., Chen, N., Dang, Y., Li, J., Yang, C., Chen, W., Su, Y., Cong, X., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.07924 · Cited by 81
Emergent autonomous scientific research capabilities of large language models
Boiko, D. A., MacKnight, R., & Gomes, G. (2023) · arXiv · doi:10.48550/arxiv.2304.05332 · Cited by 72
The Semantic Scholar Open Data Platform
Kinney, R., Anastasiades, C., Authur, R., Beltagy, I., Bragg, J., Buraczynski, A., Cachola, I., Candra, S., Chandrasekhar, Y., Cohan, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.10140 · Cited by 64
AI intensifies fight against ‘paper mills’ that churn out fake research
Liverpool, L. (2023) · Nature · doi:10.1038/d41586-023-01780-w · Cited by 63
Steinruecken, C., Smith, E., Janz, D., Lloyd, J., & Ghahramani, Z. (2019) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-030-05318-5_9 · Cited by 46
DERA: Enhancing large language model completions with dialog-enabled resolving agents
Nair, V., Schumacher, E., Tso, G., & Kannan, A. (2024) · Proceedings of the 6th Clinical Natural Language Processing Workshop · doi:10.18653/v1/2024.clinicalnlp-1.12 · Cited by 38
Flury, B. B., Güsewell, S., Egger, T., Leal-Neto, O. B., Brucher, A., Lemmenmeier, E., Kleeb, D. M., Möller, J. C., Rieder, P., Ruetti, M., et al. (2022) · PLoS Medicine · doi:10.1371/journal.pmed.1004125 · Cited by 27
A Congressional Twitter network dataset quantifying pairwise probability of influence
Fink, C. G., Omodt, N., Zinnecker, S., & Sprint, G. (2023) · Data in Brief · doi:10.1016/j.dib.2023.109521 · Cited by 25
Shim, J., Ryu, K., Lee, S. H., Cho, E.-A., Lee, S., & Ahn, J. H. (2021) · PLoS ONE · doi:10.1371/journal.pone.0257069 · Cited by 18
Hundreds of gibberish papers still lurk in the scientific literature
Van Noorden, R. (2021) · Nature · doi:10.1038/d41586-021-01436-7 · Cited by 16
Solving Math Word Problems by Combining Language Models With Symbolic Solvers
He-Yueya, J., Poesia, G., Wang, R. E., & Goodman, N. D. (2023) · arXiv · doi:10.48550/arxiv.2304.09102 · Cited by 16
Saint‐Fleur, A. L., Alcalá, H. E., & Sridhar, S. (2023) · PLoS ONE · doi:10.1371/journal.pone.0289945 · Cited by 12
RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text
Zhou, W., Jiang, Y. E., Cui, P., Wang, T., Xiao, Z., Hou, Y., Cotterell, R., & Sachan, M. (2023) · arXiv · doi:10.48550/arxiv.2305.13304 · Cited by 9
Research works printed but not linked: 14
- Tu T Palepu A Schaekermann M et al. Towards conversational diagnostic AI. January 11 2024 (http://arXiv.org/abs/2401.05654). Preprint. · arXiv
- 10.1056/AIoa2400196 · doi:10.1056/aioa2400196 · Paper page
- Wilson EB. An introduction to scientific research. McGraw-Hill Book Company, 1952.
- Ji Z Lee N Frieske R et al. Survey of hallucination in natural language generation. February 8 2022 (http://arXiv.org/abs/2202.03629). Preprint. · arXiv
- Touvron H Martin L Stone K et al. Llama 2: open foundation and fine-tuned chat models. July 18 2023 (http://arXiv.org/abs/2307.09288). Preprint. · arXiv
- Rozière B Gehring J Gloeckle F et al. Code Llama: open foundation models for code. August 24 2023 (http://arXiv.org/abs/2308.12950). Preprint. · arXiv
- Wu Q Bansal G Zhang J et al. AutoGen: enabling next-gen LLM applications via multi-agent conversation. August 16 2023 (http://arXiv.org/abs/2308.08155). Preprint. · arXiv
- Hong S Zhuge M Chen J et al. MetaGPT: meta programming for a multi-agent collaborative framework. August 1 2023 (http://arXiv.org/abs/2308.00352). Preprint. · arXiv
- Madaan A Tandon N Gupta P et al. Self-refine: iterative refinement with self-feedback. March 30 2023 (http://arXiv.org/abs/2303.17651). Preprint. · arXiv
- Rolle-Lake L, Robbins E. Behavioral risk factor surveillance system. In: StatPearls. Treasure Island, FL: StatPearls Publishing, 2023 (https://www.ncbi.nlm.nih.gov/pubmed/31971707).
- Majumder BP Surana H Agarwal D et al. Data-driven discovery with large generative models. February 21 2024 (http://arXiv.org/abs/2402.13610). Preprint. · arXiv
- Abdel-Rehim A Zenil H Orhobor O et al. Scientific hypothesis generation by a large language model: laboratory validation in breast cancer treatment. May 20 2024 (http://arXiv.org/abs/2405.12258). Preprint. · arXiv
- Wang Q Downey D Ji H Hope T. SciMON: scientific inspiration machines optimized for novelty. May 23 2023 (http://arXiv.org/abs/2305.14259). Preprint. · arXiv
- Lu C Lu C Lange RT Foerster J Clune J Ha D. The AI scientist: towards fully automated open-ended scientific discovery. August 12 2024 (http://arXiv.org/abs/2408.06292). Preprint. · arXiv
Cited by
Artificial Intelligence for Materials Discovery, Development, and Optimization
Madika, B., Saha, A., Kang, C., Buyantogtokh, B., Agar, J., Wolverton, C. M., Voorhees, P., Littlewood, P., Kalinin, S., & Hong, S. (2025) · ACS Nano · doi:10.1021/acsnano.5c04200 · Cited by 171
Towards end-to-end automation of AI research
Lu, C., Lu, C., Lange, R. T., Yamada, Y., Hu, S., Foerster, J., Ha, D., & Clune, J. (2026) · Nature · doi:10.1038/s41586-026-10265-5 · Cited by 116
A multi-agent system for automating scientific discovery
Ghareeb, A. E., Chang, B., Mitchener, L., Angela, Y., Szostkiewicz, C. J., Shved, D., Gyimesi, G. J., Laurent, J. M., Wright, S. M., Razzak, M. T., et al. (2026) · Nature · doi:10.1038/s41586-026-10652-y · Cited by 454
How can artificial intelligence transform the training of medical students and physicians?
Ning, Y., Ong, J. C. L., Cheng, H., Wang, H., Ting, D. S. W., Tham, Y. C., Wong, T. Y., & Liu, N. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.100900 · Cited by 77
Responsible AI in biotechnology: balancing discovery, innovation and biosecurity risks
Wheeler, N. E. (2025) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2025.1537471 · Cited by 71
Metadata
- Authors
- Ifargan, T., Hafner, L., Kern, M., Alcalay, O., & Kishony, R.
- Affiliations
- Technion – Israel Institute of Technology (IL)
- Epsio, Tel Aviv, Israel
- Venue
- NEJM AI
- Publisher
- Massachusetts Medical Society
- Published
- 2025-01-01
- Volume
- 2
- Issue
- 1
- Pages
- AIoa2400555-AIoa2400555
- ISSN
- 2836-9386 (ISSN-L)
- OpenAlex
- W4404941741
- Cited by
- 34
- References
- 26
- Citations by year
- 2026: 13 · 2025: 20
- FWCI
- 50.38
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- NEJM AI (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Keywords
- large language models, autonomous research, human verification, data-driven research, AI-assisted scientific writing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.