CoChat

Autonomous LLM-Driven Research — from Data to Human-Verifiable Research Papers

Open access
Ifargan, Tal, Hafner, Lukas, Kern, Maor, Alcalay, Ori, Kishony, Roy · 2025 · NEJM AI

The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.

Abstract

BACKGROUND: Artificial intelligence (AI) promises to accelerate scientific discovery, but it remains unclear whether AI systems can perform fully autonomous research, and whether they can do so while adhering to key scientific values, such as transparency, traceability, and verifiability. The aim of this study was to develop and evaluate an AI-automation platform that performs transparent, traceable, and human-verifiable scientific research. METHODS: To mimic human scientific practices, we built "data-to-paper," an automation platform that guides interacting large language model (LLM) agents through a complete stepwise research process that starts with annotated data and results in comprehensive research papers, while programmatically backtracing information flow and allowing human oversight and interactions. The platform can run fully autonomously (in autopilot mode) or with human intervention (in copilot mode). RESULTS: In autopilot mode, provided only with annotated data, data-to-paper raised hypotheses; designed research plans; wrote and debugged analysis codes; generated and interpreted results; and created complete, information-traceable research papers. Even though the research novelty of manuscripts created by data-to-paper was relatively limited, the process demonstrated the autonomous generation of de novo quantitative insights from data, such as unraveling associations between health indicators and clinical outcomes. For simple research goals and datasets, a fully autonomous cycle can create manuscripts that independently recapitulate the findings of peer-reviewed biomedical publications without major errors in about 80 to 90% of cases. Yet, as goal or data complexity increases, human copiloting becomes critical for ensuring accuracy and overall quality. By tracking information flow through the steps, the platform creates "data-chained" manuscripts, in which downstream results are programmatically linked to upstream code and data, thus setting a new standard for the verifiability of scientific outputs. CONCLUSIONS: Our work demonstrates the potential for AI-driven acceleration of scientific discovery in data-driven biomedical research and beyond, while enhancing, rather than jeopardizing, traceability, transparency, and verifiability.

Overview

Integrity signals

Registry

Crossref · 10.1056/aioa2400555

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 38 references

Research works printed but not linked: 14

  1. Tu T Palepu A Schaekermann M et al. Towards conversational diagnostic AI. January 11 2024 (http://arXiv.org/abs/2401.05654). Preprint. · arXiv
  2. 10.1056/AIoa2400196 · doi:10.1056/aioa2400196 · Paper page
  3. Wilson EB. An introduction to scientific research. McGraw-Hill Book Company, 1952.
  4. Ji Z Lee N Frieske R et al. Survey of hallucination in natural language generation. February 8 2022 (http://arXiv.org/abs/2202.03629). Preprint. · arXiv
  5. Touvron H Martin L Stone K et al. Llama 2: open foundation and fine-tuned chat models. July 18 2023 (http://arXiv.org/abs/2307.09288). Preprint. · arXiv
  6. Rozière B Gehring J Gloeckle F et al. Code Llama: open foundation models for code. August 24 2023 (http://arXiv.org/abs/2308.12950). Preprint. · arXiv
  7. Wu Q Bansal G Zhang J et al. AutoGen: enabling next-gen LLM applications via multi-agent conversation. August 16 2023 (http://arXiv.org/abs/2308.08155). Preprint. · arXiv
  8. Hong S Zhuge M Chen J et al. MetaGPT: meta programming for a multi-agent collaborative framework. August 1 2023 (http://arXiv.org/abs/2308.00352). Preprint. · arXiv
  9. Madaan A Tandon N Gupta P et al. Self-refine: iterative refinement with self-feedback. March 30 2023 (http://arXiv.org/abs/2303.17651). Preprint. · arXiv
  10. Rolle-Lake L, Robbins E. Behavioral risk factor surveillance system. In: StatPearls. Treasure Island, FL: StatPearls Publishing, 2023 (https://www.ncbi.nlm.nih.gov/pubmed/31971707).
  11. Majumder BP Surana H Agarwal D et al. Data-driven discovery with large generative models. February 21 2024 (http://arXiv.org/abs/2402.13610). Preprint. · arXiv
  12. Abdel-Rehim A Zenil H Orhobor O et al. Scientific hypothesis generation by a large language model: laboratory validation in breast cancer treatment. May 20 2024 (http://arXiv.org/abs/2405.12258). Preprint. · arXiv
  13. Wang Q Downey D Ji H Hope T. SciMON: scientific inspiration machines optimized for novelty. May 23 2023 (http://arXiv.org/abs/2305.14259). Preprint. · arXiv
  14. Lu C Lu C Lange RT Foerster J Clune J Ha D. The AI scientist: towards fully automated open-ended scientific discovery. August 12 2024 (http://arXiv.org/abs/2408.06292). Preprint. · arXiv

Cited by

See all 38

Metadata

Authors
Ifargan, T., Hafner, L., Kern, M., Alcalay, O., & Kishony, R.
Affiliations
  • Technion – Israel Institute of Technology (IL)
  • Epsio, Tel Aviv, Israel
Venue
NEJM AI
Published
2025-01-01
Volume
2
Issue
1
Pages
AIoa2400555-AIoa2400555
ISSN
2836-9386 (ISSN-L)
OpenAlex
W4404941741
Cited by
34
References
26
Citations by year
2026: 13 · 2025: 20
FWCI
50.38
Type
journal article
Language
en
Open access
Locations
  • NEJM AI (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Social Sciences › Library and Information Sciences › Academic Writing and Publishing
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
large language models, autonomous research, human verification, data-driven research, AI-assisted scientific writing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Autonomous LLM-Driven Research — from Data to Human-Verifiable Research Papers. https://app.cochat.ai/papers/10.1056/aioa2400555
The paper
Ifargan, T., Hafner, L., Kern, M., Alcalay, O., & Kishony, R. (2025). Autonomous LLM-Driven Research — from Data to Human-Verifiable Research Papers. NEJM AI. https://doi.org/10.1056/aioa2400555

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.