Zero-Shot Clinical Trial Patient Matching with LLMs

Abstract

AbstractMatching patients to clinical trials is a key challenge in bringing new drugs to market. Identifying patients who meet eligibility criteria for a trial is highly manual, taking up to 1 hour per patient. Automated screening is challenging, however, as it requires the ability to understand unstructured clinical text. To address this, we have designed a zero-shot large language model (LLM)-based system that evaluates a patient's medical history (as unstructured clinical text) against trial inclusion criteria (also specified as free text). We investigate different prompting strategies and design a novel two-stage retrieval pipeline to reduce the number of tokens processed by up to a third while sustaining high performance. Our contributions are threefold. First, we achieve state-of-the-art performance on the 2018 n2c2 cohort selection challenge, the largest public benchmark for clinical trial patient matching. Second, this system can improve the data and cost efficiency of matching patients an order of magnitude faster and more affordably than the status quo. Third, we demonstrate the interpretability of our system by generating natural language justifications for each eligibility decision, which clinicians found coherent in 97% of correct decisions and 75% of incorrect ones. These results establish the feasibility of using LLMs to accelerate clinical trial operations, with the zero-shot retrieval architecture scalable to arbitrary trials and patient record length with minimal reconfiguration. (Funded by the Clinical Excellence Research Center at Stanford Medicine and others.).