Cited by 309Artificial Intelligence in Healthcare and EducationMachine Learning in HealthcareInformation Retrieval and Search Behavior

Matching patients to clinical trials with large language models

Open access

Abstract

Patient recruitment is challenging for clinical trials. We introduce TrialGPT, an end-to-end framework for zero-shot patient-to-trial matching with large language models. TrialGPT comprises three modules: it first performs large-scale filtering to retrieve candidate trials (TrialGPT-Retrieval); then predicts criterion-level patient eligibility (TrialGPT-Matching); and finally generates trial-level scores (TrialGPT-Ranking). We evaluate TrialGPT on three cohorts of 183 synthetic patients with over 75,000 trial annotations. TrialGPT-Retrieval can recall over 90% of relevant trials using less than 6% of the initial collection. Manual evaluations on 1015 patient-criterion pairs show that TrialGPT-Matching achieves an accuracy of 87.3% with faithful explanations, close to the expert performance. The TrialGPT-Ranking scores are highly correlated with human judgments and outperform the best-competing models by 43.8% in ranking and excluding trials. Furthermore, our user study reveals that TrialGPT can reduce the screening time by 42.6% in patient recruitment. Overall, these results have demonstrated promising opportunities for patient-to-trial matching with TrialGPT.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41467-024-53081-z

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Many published trial-matching approaches remain retrospective, disease-limited, or optimized for text similarity and retrieval rather than auditable patient-trial execution over longitudinal records.

    “many published approaches remain retrospective, disease-limited, or optimized for text similarity and retrieval rather than auditable patient–trial execution over longitudinal records.”

    From Transforming oncology clinical trial matching through neuro-symbolic, multi-agent AI and an oncology-specific knowledge graph: a prospective evaluation in 3804 patients (2026) · cites this paper as [4]

    The Introduction describes prior embedding-based similarity search as non-explainable, and the Discussion says this study did not evaluate longitudinal records or EHR integration. The paper’s evaluation uses existing datasets, but these passages do not establish that many published approaches are retrospective or disease-limited. (LLM, conf=0.83)

    Experimental — accuracy not yet measured
  • TrialGPT uses a trio of agents to match patients to clinical trials.

    “A pioneering system in this realm is TrialGPT, which uses a trio of agents to match patients to clinical trials [36].”

    From A Review of Multi-Agent AI Systems for Biological and Clinical Data Analysis (2026) · cites this paper as [36]

    The Methods section (“TrialGPT”) describes a patient–trial matching architecture with three modules: Retrieval, Matching, and Ranking. This supports the three-part structure and purpose, but the paper calls them modules, not agents; it does not establish that TrialGPT uses three agents. (LLM, conf=0.85)

    Experimental — accuracy not yet measured
  • Clinicians assisted by TrialGPT completed trial screening tasks 42.6% faster than those working without AI assistance (35.3 s vs. 61.5 s; p < 0.001).

    “clinicians assisted by TrialGPT were able to complete trial screening tasks 42.6% faster than those working without AI assistance (35.3 s vs. 61.5 s; p < 0.001)”

    From A Review of Multi-Agent AI Systems for Biological and Clinical Data Analysis (2026) · cites this paper as [36]

    The Results section, ‘TrialGPT can reduce the screening time for patient-trial matching’ (page not provided), reports about 42.6% less screening time with TrialGPT in a pilot study involving two medical doctors. Our copy does not state the claimed overall times of 35.3 s and 61.5 s or a p-value below 0.001 for that comparison. (LLM, conf=0.86)

    Experimental — accuracy not yet measured
  • The high correlation between TrialGPT's ranking scores and human judgment suggests that MAS has the potential to serve as a reliable cognitive assistant for complex, zero-shot patient-to-trial matching.

    “the high correlation between TrialGPT’s ranking scores and human judgment suggests that MAS has the potential to serve as a reliable cognitive assistant for complex, zero-shot patient-to-trial matching”

    From A Review of Multi-Agent AI Systems for Biological and Clinical Data Analysis (2026) · cites this paper as [36]

    The Results section reports that TrialGPT-Ranking scores correlate highly with patient–trial eligibility and that its LLM-aggregated scores correlate significantly with manual eligibility labels. The Discussion supports using TrialGPT to assist human recruiters, but the paper does not evaluate a multi-agent system (MAS); extending the finding to MAS is the citing authors’ inference, not a finding of this source. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • TrialGPT's AUROC of 0.7979 represents a 43.8% improvement over cross-encoder BioLinkBERT (AUROC 0.6176).

    “representing a 43.8% improvement over state-of-the-art baselines like the cross-encoder BioLinkBERT (AUROC: 0.6176)”

    From A Review of Multi-Agent AI Systems for Biological and Clinical Data Analysis (2026) · cites this paper as [36]

    In Results (section “TrialGPT-Ranking can effectively rank and exclude candidate clinical trials”), TrialGPT’s AUROC is 0.7979 and cross-encoder BioLinkBERT’s is 0.6176. The reported 43.8% improvement refers instead to the overall average score (0.7314 versus 0.5085); the AUROC improvement over 0.6176 is about 29.2%. (LLM, conf=0.94)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 7

  1. [8]Voorhees, E. M. & Hersh, W. R. Overview of the TREC 2012 Medical Records Track. In Proc. Twenty-First Text REtrieval Conference (TREC 2012) (2012).
  2. [9]Roberts, K., Demner-Fushman, D., Voorhees, E. M., Bedrick, S. & Hersh, W. R. Overview of the TREC 2022 Clinical Trials Track. In Proc. Thirty-First Text REtrieval Conference (TREC 2022) (2022). PMC8629152 34849513
  3. [12]Jin, Q., Tan, C., Zhao, Z., Yuan, Z. & Huang, S. Alibaba DAMO Academy at TREC Clinical Trials 2021: Exploring Embedding-based First-stage Retrieval with TrialMatcher. In Proc. Thirtieth Text REtrieval Conference (TREC 2021) (2021).
  4. [13]Roberts, K., Demner-Fushman, D., Voorhees, E. M., Bedrick, S. & Hersh, W. R. Overview of the TREC 2021 Clinical Trials Track. In Proc. Thirtieth Text REtrieval Conference (TREC 2021) (2021). PMC8629152 34849513
  5. [38]Roberts, K., Demner-Fushman, D., Voorhees, E. M. & Hersh, W. R. Overview of the TREC 2014 Clinical Decision Support Track. In Proc. Twenty-Third Text REtrieval Conference (TREC 2014) (2014).
  6. [39]Roberts, K., Simpson, M. S., Voorhees, E. M. & Hersh, W. R. Overview of the TREC 2015 Clinical Decision Support Track. In Proc. Twenty-Fourth Text REtrieval Conference (TREC 2015) (2015).
  7. [44]Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) 4171–4186 (2019).

Cited by

Sorted by citations

Authors
Jin, Q., Wang, Z., Floudas, C. S., Chen, F., Gong, C., Bracken-Clarke, D., Xue, E., Yang, Y., Sun, J., & Lu, Z.
Affiliations
  • National Institutes of Health (US)
  • University of Illinois Urbana-Champaign (US)
  • University of Pittsburgh (US)
  • Albert Einstein College of Medicine (US)
Corresponding author
Lu, Zhiyong
Venue
Nature Communications
Published
2024-11-18
Volume
15
Issue
1
Pages
9074-9074
ISSN
2041-1723 (ISSN-L)
OpenAlex
W4404486407
Cited by
287
References
43
Citations by year
2026: 119 · 2025: 117 · 2024: 50 · 2023: 1
FWCI
26.74
Type
journal article
Language
en
Open access
Locations
  • Nature Communications (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • PubMed Central (submittedVersion, cc-by)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University) (cc-by)
Funders
U.S. Department of Health & Human Services | National Institutes of Health
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
Keywords
clinical trial recruitment, large language models, explainable AI, synthetic patient data, user study

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review