End-To-End Clinical Trial Matching with Large Language Models
Open accessAbstract
Matching cancer patients to clinical trials is essential for advancing treatment and patient care. However, the inconsistent format of medical free text documents and complex trial eligibility criteria make this process extremely challenging and time-consuming for physicians. We investigated whether the entire trial matching process - from identifying relevant trials among 105,600 oncology-related clinical trials on clinicaltrials.gov to generating criterion-level eligibility matches - could be automated using Large Language Models (LLMs). Using GPT-4o and a set of 51 synthetic Electronic Health Records (EHRs), we demonstrate that our approach identifies relevant candidate trials in 93.3% of cases and achieves a preliminary accuracy of 88.0% when matching patient-level information at the criterion level against a baseline defined by human experts. Utilizing LLM feedback reveals that 39.3% criteria that were initially considered incorrect are either ambiguous or inaccurately annotated, leading to a total model accuracy of 92.7% after refining our human baseline. In summary, we present an end-to-end pipeline for clinical trial matching using LLMs, demonstrating high precision in screening and matching trials to individual patients, even outperforming the performance of qualified medical doctors. Our fully end-to-end pipeline can operate autonomously or with human supervision and is not restricted to oncology, offering a scalable solution for enhancing patient-trial matching in real-world settings.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
One study used a similar "filter-and-apply" pipeline, in which trials were first filtered and then matched to patients, showcasing the effectiveness of this approach.
“One study used a similar “filter-and-apply” pipeline, in which trials were first filtered and then matched to patients, showcasing the effectiveness of this approach [35].”
From Identifying Deprescribing Opportunities With Large Language Models in Older Adults: Retrospective Cohort Study (2025) · cites this paper as [35]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 18
- Bouzalmate-Hajjaj, A., Massó Guijarro, P., Khan, K. S., Bueno-Cavanillas, A. & Cano-Ibáñez, N. Benefits of Participation in Clinical Trials: An Umbrella Review. Int. J. Environ. Res. Public Health 19, (2022).
- Unger, J. M., Vaidya, R., Hershman, D. L., Minasian, L. M. & Fleury, M. E. Systematic Review and Meta-Analysis of the Magnitude of Structural, Clinical, and Physician and Patient Barriers to Cancer Clinical Trial Participation. J. Natl. Cancer Inst. 111, 245-255 (2019).
- Rule, A., Bedrick, S., Chiang, M. F. & Hribar, M. R. Length and Redundancy of Outpatient Progress Notes Across a Decade at an Academic Medical Center. JAMA Netw Open 4, e2115334 (2021).
- Moy, A. J. et al. Measurement of clinical documentation burden among physicians and nurses using electronic health records: a scoping review. J. Am. Med. Inform. Assoc. 28, 998-1008 (2021).
- Bradley, J., Kelly, K. & Stinchcombe, T. E. The Ever-Increasing Number of Trial Eligibility Criteria: Time to Bend the Curve. Journal of thoracic oncology: official publication of the International Association for the Study of Lung Cancer vol. 12 1459-1460 (2017).
- Zhang, X., Xiao, C., Glass, L. M. & Sun, J. DeepEnroll: Patient-Trial Matching with Deep Embedding and Entailment Prediction. arXiv [cs.AI] (2020).
- Gao, J., Xiao, C., Glass, L. M. & Sun, J. COMPOSE: Cross-Modal Pseudo- Siamese Network for Patient Trial Matching. in Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining 803-812 (Association for Computing Machinery, New York, NY, USA, 2020).
- OpenAI et al. GPT-4 Technical Report. arXiv [cs.CL] (2023).
- Jin, Q. et al. Matching Patients to Clinical Trials with Large Language Models. ArXiv (2024).
- den Hamer, D. M., Schoor, P., Polak, T. B. & Kapitan, D. Improving Patient Pre-screening for Clinical Trials: Assisting Physicians with Large Language Models. arXiv [cs.LG] (2023).
- Wong, C. et al. Scaling Clinical Trial Matching Using Large Language Models: A Case Study in Oncology. in Proceedings of the 8th Machine Learning for Healthcare Conference (eds.
- Deshpande, K. et al.) vol. 219 846-862 (PMLR, 11--12 Aug 2023).
- Gupta, S. K. et al. PRISM: Patient Records Interpretation for Semantic Clinical Trial Matching using Large Language Models. arXiv [cs.CL] (2024).
- Wornow, M. et al. Zero-Shot Clinical Trial Patient Matching with LLMs. arXiv [cs.CL] (2024).
- Singhvi, A. et al. DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines. arXiv [cs.CL] (2023).
- Cattell, R. Scalable SQL and NoSQL data stores. SIGMOD Rec. 39, 12-27 (2011).
- Benary, M. et al. Leveraging Large Language Models for Decision Support in Personalized Oncology. JAMA Netw Open 6, e2343689 (2023).
- Sasazawa, Y., Yokote, K., Imaichi, O. & Sogawa, Y. Text Retrieval with Multi-Stage Re-Ranking Models. arXiv [cs.IR] (2023). Hepatic Lesions: Slight growth and increased metabolic activity. Lymph Nodes: Slightly re-enlarged hilar and subcarinal nodes. Pleural Effusion: Small right-sided pleural effusion noted. Atelectasis: Slight worsening of minor atelectasis in the left lower lobe. Adrenal Gland: Remains slightly enlarged; increased metabolic activity. Overall Assessment: Disease progression (PD). May 15, 2024: Tumor board recommends considering clinical trial options due to limited response to standard therapies. May 18, 2024: Detailed assessment of health status. ECOG performance 1. All routine labs, including liver and renal function tests, within normal limits. ===== Patient 10.1.1 ===== Name: Miller, Jane Born: 25.07.1965 Address: Main Street 78, Potsdam June 5, 2021: CT scan of the abdomen and pelvis: Revealed a well-defined mass in the head of the pancreas measuring approximately 4.5 cm with surrounding lym- phadenopathy. No signs of vascular invasion, but multiple small hepatic lesions were identified, indicating metastatic disease. No evidence of bowel obstruction, but slight dilation of the pancreatic duct was observed. July 1, 2021: Biopsy and molecular testing confirmed a well-differentiated neuroen- docrine tumor with a high tumor mutational burden (TMB-H, >=10 mut/Mb, F1CDx assay), Grade 3. Initiated Capecitabine and Temozolomide chemotherapy regimen on July 20, 2021. October 10, 2021: Follow-up CT scan (chest, abdomen, and pelvis): Partial response to treatment observed, with the primary pancreatic mass reducing to 3.5 cm in diameter. Hepatic lesions showed slight reduction in size and metabolic activity. No new metastatic lesions were detected, but mild ascites persisted. January 1, 2022: Continued Capecitabine and Temozolomide therapy, with staging scans showing moderate disease progression. Primary mass increased to 4.0 cm, with new peritoneal nodules. February 1, 2022: Initiated Everolimus therapy due to progression on previous regimen. August 1, 2022: Follow-up MRI scan (abdomen and pelvis): MRI indicated further disease progression, with the primary tumor enlarging to 4.8 cm and increased involve- ment of adjacent hepatic structures. Peritoneal nodules showed slight growth, and moderate ascites was present. There was no evidence of bowel obstruction or significant vascular invasion. November 15, 2022: Tumor board recommends considering clinical trial options due to limited response to standard therapies. December 1, 2022: Detailed assessment of health status. ECOG performance status
Cited by
Sorted by citations
Artificial intelligence-based biomarkers for treatment decisions in oncology
Ligero, M., Nahhas, O. S. M. E., Aldea, M. D., & Kather, J. N. (2025) · Trends in cancer · doi:10.1016/j.trecan.2024.12.001 · Cited by 70 · see citing works
Self-citationESMO guidance on the use of Large Language Models in Clinical Practice (ELCAP)
Wong, E. Y. T., Verlingue, L., Aldea, M., Franzoi, M. A., Umeton, R., Halabi, S., Harbeck, N., Indini, A., Prelaj, A., Romano, E., et al. (2025) · Annals of Oncology · doi:10.1016/j.annonc.2025.09.001 · Cited by 44 · see citing works
Self-citationThe Use of Artificial Intelligence for Cancer Therapeutic Decision-Making
Elemento, O., Khozin, S., & Sternberg, C. N. (2025) · NEJM AI · doi:10.1056/aira2401164 · Cited by 28 · see citing works
AI and innovation in clinical trials
Badani, A., de Moraes, F. Y., Vollmuth, P., Chung, C. D., & Mansouri, A. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02048-5 · Cited by 33 · see citing works
The barriers to uptake of artificial intelligence in hepatology and how to overcome them
Clusmann, J., Balaguer‐Montero, M., Bassegoda, O., Schneider, C. V., Seraphin, T. P., Paintsil, E. K., Luedde, T., Pérez-López, R., Caldéraro, J., Gilbert, S., et al. (2025) · Journal of Hepatology · doi:10.1016/j.jhep.2025.07.003 · Cited by 29 · see citing works
Self-citationRetractedArtificial intelligence agents in cancer research and oncology.
Truhn, D., Azizi, S., Zou, J., Cerdá-Alberich, L., Mahmood, F., & Kather, J. (2026) · Nature Reviews. Cancer · doi:10.1038/s41568-025-00900-0 · Cited by 27 · see citing works
Self-citationSocrates, V., Wright, D. S., Huang, T., Fereydooni, S., Dien, C., Chi, L., Albano, J., Patterson, B., Sasidhar Kanaparthy, N., Wright, C. X., et al. (2025) · JMIR Aging · doi:10.2196/69504 · Cited by 17 · see citing works
A prospective pragmatic evaluation of automatic trial matching tools in a molecular tumor board
Gueguen, L., Olgiati, L., Brutti-Mairesse, C., Sans, A., Texier, V. L., & Verlingue, L. (2025) · npj Precision Oncology · doi:10.1038/s41698-025-00806-y · Cited by 14 · see citing works
Chen, H., Li, X., He, X., Chen, A., McGill, J. M., Webber, E. C., Xu, H., Liu, M., & Bian, J. (2025) · JCO Clinical Cancer Informatics · doi:10.1200/cci-25-00071 · Cited by 14 · see citing works
Large language models for automating clinical trial matching
Layne, E., Olivas, C. G., Hershenhouse, J. S., Ganjavi, C., Cei, F., Gill, I. S., & Cacciamani, G. E. (2025) · Current Opinion in Urology · doi:10.1097/mou.0000000000001281 · Cited by 6 · see citing works
Use of Large Language Models in Clinical Cancer Research
Kehl, K. L. (2025) · JCO Clinical Cancer Informatics · doi:10.1200/cci-25-00027 · Cited by 8 · see citing works
How AI will transform cancer care
Kather, J. N., & Abernethy, A. P. (2024) · Annals of Oncology · doi:10.1016/j.annonc.2024.08.2335 · Cited by 4 · see citing works
Self-citation
- Authors
- Ferber, D., Hilgers, L., Wiest, I. C., Leßmann, M.-E., Clusmann, J., Neidlinger, P., Zhu, J., Wölflein, G., Lammert, J., Tschochohei, M., Böhme, H., Jäger, D., Aldea, M., Truhn, D., Höper, C., & Kather, J. N.
- Affiliations
- Heidelberg University (DE)
- Université Paris-Saclay (FR)
- Else Kröner Fresenius Center for Digital Health (DE)
- TUM Klinikum (DE)
- Institut Gustave Roussy (FR)
- Universitätsklinikum Aachen (DE)
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2407.13463
- OpenAlex
- W4402345273
- Cited by
- 12
- References
- 0
- Citations by year
- 2026: 1 · 2025: 10 · 2024: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Funders
- AstraZeneca
- National Institute for Health and Care Research (NIHR203331)
- Department of Health and Social Care
- European Commission (101096312, 101057091)
- Deutscher Akademischer Austauschdienst (57616814)
- Bundesministerium für Bildung und Forschung (01KD2215A, 01EO2101, 01VSF21048, 57616814, 01KD2215B, 01KD2104C, 031L0312A)
- Bundesministerium für Gesundheit
- Deutsche Krebshilfe (70115166)
- Gemeinsame Bundesausschuss (01VSF21048)
- National Institutes of Health
- HORIZON EUROPE Framework Programme (101057091)
- Leeds Biomedical Research Centre (NIHR203331)
- National Cancer Institute (R01CA263318)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Keywords
- clinical trial matching, large language models, GPT-4o, cancer patients, inclusion and exclusion criteria, ClinicalTrials.gov, electronic health records, cancer clinical trials
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
