Cited by 142Electronic Health Records SystemsArtificial Intelligence in Healthcare and EducationBiomedical Text Mining and Ontologies

Increasing the efficiency of trial-patient matching: automated clinical trial eligibility Pre-screening for pediatric oncology patients

Open access
Ni, Yizhao, Wright, Jordan, Perentesis, John, Lingren, Todd, Deleger, Louise, Kaiser, Megan, Kohane, Isaac, Solti, Imre·2015·BMC Medical Informatics and Decision Making

Abstract

BACKGROUND: Manual eligibility screening (ES) for a clinical trial typically requires a labor-intensive review of patient records that utilizes many resources. Leveraging state-of-the-art natural language processing (NLP) and information extraction (IE) technologies, we sought to improve the efficiency of physician decision-making in clinical trial enrollment. In order to markedly reduce the pool of potential candidates for staff screening, we developed an automated ES algorithm to identify patients who meet core eligibility characteristics of an oncology clinical trial. METHODS: We collected narrative eligibility criteria from ClinicalTrials.gov for 55 clinical trials actively enrolling oncology patients in our institution between 12/01/2009 and 10/31/2011. In parallel, our ES algorithm extracted clinical and demographic information from the Electronic Health Record (EHR) data fields to represent profiles of all 215 oncology patients admitted to cancer treatment during the same period. The automated ES algorithm then matched the trial criteria with the patient profiles to identify potential trial-patient matches. Matching performance was validated on a reference set of 169 historical trial-patient enrollment decisions, and workload, precision, recall, negative predictive value (NPV) and specificity were calculated. RESULTS: Without automation, an oncologist would need to review 163 patients per trial on average to replicate the historical patient enrollment for each trial. This workload is reduced by 85% to 24 patients when using automated ES (precision/recall/NPV/specificity: 12.6%/100.0%/100.0%/89.9%). Without automation, an oncologist would need to review 42 trials per patient on average to replicate the patient-trial matches that occur in the retrospective data set. With automated ES this workload is reduced by 90% to four trials (precision/recall/NPV/specificity: 35.7%/100.0%/100.0%/95.5%). CONCLUSION: By leveraging NLP and IE technologies, automated ES could dramatically increase the trial screening efficiency of oncologists and enable participation of small practices, which are often left out from trial enrollment. The algorithm has the potential to significantly reduce the effort to execute clinical research at a point in time when new initiatives of the cancer care community intend to greatly expand both the access to trials and the number of available trials.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12911-015-0149-3

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • BERT-based tools have been used for subject matching across various diseases while struggling with generalizability.

    “Bidirectional Encoder Representations from Transformers (BERT)-based tools have been used for subject matching across various diseases while struggling with generalizability [4–7].”

    From Enhancing hepatopathy clinical trial efficiency: a secure, large language model-powered pre-screening pipeline (2025) · cites this paper as [4]

    The Methods (Automated ES algorithm; page unavailable) describe matching patients to clinical trials, but the tool uses cTAKES and term-based vectors, not BERT. The Discussion (Error analysis, limitations and future work; page unavailable) says the study was restricted to pediatric oncology and that transferability to other populations needs testing. It does not establish use across various diseases. (LLM, conf=0.88)

    Experimental — accuracy not yet measured
  • Traditional manual screening and automated approaches fall short in key areas.

    “While traditional manual screening and automated approaches have attempted to address these challenges, they fall short in key areas [4].”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 35 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

  1. [29]Qi Y Laquerre PF Retrieving medical records with "sennamed": NEC labs America at TREC 2012 medical records track The twenty first Text REtrieval Conference Proceedings (TREC) 2012
  2. [33]Zhai H Iyer S Ni Y Lingren T Kirkendall E Li Q Mining a large-scale EHR with machine learning methods to predict all-cause 30-day unplanned readmissions Proc. of the 2nd ASE International Conference on Big Data Science and Computing 2014

Cited by

Sorted by citations

Authors
Ni, Y., Wright, J., Perentesis, J., Lingren, T., Deleger, L., Kaiser, M., Kohane, I., & Solti, I.
Affiliations
  • Cincinnati Children's Hospital Medical Center (US)
  • Harvard University (US)
Corresponding author
Ni, Yizhao
Venue
BMC Medical Informatics and Decision Making
Published
2015-04-14
Volume
15
Issue
1
Pages
28-28
ISSN
1472-6947 (ISSN-L)
OpenAlex
W2116972869
Cited by
142
References
35
Citations by year
2026: 13 · 2025: 16 · 2024: 20 · 2023: 11 · 2022: 12 · 2021: 21 · 2020: 17 · 2019: 13 · 2018: 8 · 2017: 5 · 2016: 5 · 2015: 1
FWCI
12.20
Type
journal article
Language
en
Open access
Locations
  • BMC Medical Informatics and Decision Making (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
Funders
  • Cincinnati Children's Hospital Medical Center
  • National Institutes of Health (UL1TR001425, UL1TR000077, 1r21hd072883-01)
  • National Human Genome Research Institute (U01HG006828)
  • U.S. National Library of Medicine (R00LM010227)
  • National Center for Advancing Translational Sciences
  • Eunice Kennedy Shriver National Institute of Child Health and Human Development (R21 HD072883, 1R21HD072883-01)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
Keywords
clinical trial eligibility screening, pediatric oncology, natural language processing, information extraction, electronic health records, clinical trial recruitment, physician workload reduction
MeSH terms
Child, Clinical Trials as Topic, Eligibility Determination, Humans, Natural Language Processing, Neoplasms, Information Storage and Retrieval, Patient Selection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review