CoChat
Sign Up
Cited by 142Electronic Health Records SystemsArtificial Intelligence in Healthcare and EducationBiomedical Text Mining and Ontologies

The DOI matches its Crossref record; 34 of the 35 references it cites were found in a registry or an index.

Increasing the efficiency of trial-patient matching: automated clinical trial eligibility Pre-screening for pediatric oncology patients

Open access
Ni, Yizhao, Wright, Jordan, Perentesis, John, Lingren, Todd, Deleger, Louise, Kaiser, Megan, Kohane, Isaac, Solti, Imre·2015·BMC Medical Informatics and Decision Making
DOI 10.1186/s12911-015-0149-3PMC PMC4407835
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Manual eligibility screening (ES) for a clinical trial typically requires a labor-intensive review of patient records that utilizes many resources. Leveraging state-of-the-art natural language processing (NLP) and information extraction (IE) technologies, we sought to improve the efficiency of physician decision-making in clinical trial enrollment. In order to markedly reduce the pool of potential candidates for staff screening, we developed an automated ES algorithm to identify patients who meet core eligibility characteristics of an oncology clinical trial. METHODS: We collected narrative eligibility criteria from ClinicalTrials.gov for 55 clinical trials actively enrolling oncology patients in our institution between 12/01/2009 and 10/31/2011. In parallel, our ES algorithm extracted clinical and demographic information from the Electronic Health Record (EHR) data fields to represent profiles of all 215 oncology patients admitted to cancer treatment during the same period. The automated ES algorithm then matched the trial criteria with the patient profiles to identify potential trial-patient matches. Matching performance was validated on a reference set of 169 historical trial-patient enrollment decisions, and workload, precision, recall, negative predictive value (NPV) and specificity were calculated. RESULTS: Without automation, an oncologist would need to review 163 patients per trial on average to replicate the historical patient enrollment for each trial. This workload is reduced by 85% to 24 patients when using automated ES (precision/recall/NPV/specificity: 12.6%/100.0%/100.0%/89.9%). Without automation, an oncologist would need to review 42 trials per patient on average to replicate the patient-trial matches that occur in the retrospective data set. With automated ES this workload is reduced by 90% to four trials (precision/recall/NPV/specificity: 35.7%/100.0%/100.0%/95.5%). CONCLUSION: By leveraging NLP and IE technologies, automated ES could dramatically increase the trial screening efficiency of oncologists and enable participation of small practices, which are often left out from trial enrollment. The algorithm has the potential to significantly reduce the effort to execute clinical research at a point in time when new initiatives of the cancer care community intend to greatly expand both the access to trials and the number of available trials.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12911-015-0149-3

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 An example eligibility description (NCT01154816) derived from ClinicalTrials.gov.
Figure 1 An example eligibility description (NCT01154816) derived from ClinicalTrials.gov.CC BY · PubMed Central
Figure 2 Frequencies of the collected EHR fields (a) and descriptive statistics of the unstructured clinical notes (b). *A data entry is a piece of information (e.g. diagnosis) documented during a patient’s visit. If a patient has the same diagnosis/ICD-9 code during multiple visits, we only count the diagnosis/ICD-9 code once for that patient. **Tokens include words, numbers, symbols and punctuations in clinical narratives.
Figure 2 Frequencies of the collected EHR fields (a) and descriptive statistics of the unstructured clinical notes (b). *A data entry is a piece of information (e.g. diagnosis) documented during a patient’s visit. If a patient has the same diagnosis/ICD-9 code during multiple visits, we only count the diagnosis/ICD-9 code once for that patient. **Tokens include words, numbers, symbols and punctuations in clinical narratives.CC BY · PubMed Central
Figure 3 The architecture of the automated ES algorithm.
Figure 3 The architecture of the automated ES algorithm.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 35 references

Research works printed but not linked: 3

  1. Qi Y, Laquerre PF. Retrieving medical records with "sennamed": NEC labs America at TREC 2012 medical records track. In: The twenty first Text REtrieval Conference Proceedings (TREC). 2012. http://trec.nist.gov/pubs/trec21/papers/sennamed.medical.final.pdf (accessed 3 July 2014).
  2. Crane K. Overcoming age limits in cancer clinical trials. NCI Cancer Bull. 2010;7(10):8–9.
  3. Zhai H, Iyer S, Ni Y, Lingren T, Kirkendall E, Li Q, et al. Mining a large-scale EHR with machine learning methods to predict all-cause 30-day unplanned readmissions. In: Proc. of the 2nd ASE International Conference on Big Data Science and Computing. 2014. http://galaxy.cs.lamar.edu/~kmakki/2014-ASE/2014%20ASE%20Conference%20Stanford%20University%20Proceedings/Regular%20Full%20Paper/submission6.pdf (accessed 10 April 2015).

Cited by

Sorted by citations

Related

See all

Metadata

Authors
Ni, Y., Wright, J., Perentesis, J., Lingren, T., Deleger, L., Kaiser, M., Kohane, I., & Solti, I.
Affiliations
  • Cincinnati Children's Hospital Medical Center (US)
  • Harvard University (US)
Corresponding author
Ni, Yizhao
ORCID
Venue
BMC Medical Informatics and Decision Making
Publisher
Springer Science and Business Media LLC
Published
2015-04-14
Volume
15
Issue
1
Pages
28-28
ISSN
1472-6947 (ISSN-L)
DOI
10.1186/s12911-015-0149-3
PMC
PMC4407835
OpenAlex
W2116972869
Cited by
142
References
35
Citations by year
2026: 13 · 2025: 16 · 2024: 20 · 2023: 11 · 2022: 12 · 2021: 21 · 2020: 17 · 2019: 13 · 2018: 8 · 2017: 5 · 2016: 5 · 2015: 1
FWCI
12.20
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Informatics and Decision Making (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
Funders
  • Cincinnati Children's Hospital Medical Center
  • National Institutes of Health (UL1TR001425, UL1TR000077, 1r21hd072883-01)
  • National Human Genome Research Institute (U01HG006828)
  • U.S. National Library of Medicine (R00LM010227)
  • National Center for Advancing Translational Sciences
  • Eunice Kennedy Shriver National Institute of Child Health and Human Development (R21 HD072883, 1R21HD072883-01)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
Keywords
clinical trial eligibility screening, pediatric oncology, natural language processing, information extraction, electronic health records, clinical trial recruitment, physician workload reduction
MeSH terms
Child, Clinical Trials as Topic, Eligibility Determination, Humans, Natural Language Processing, Neoplasms, Information Storage and Retrieval, Patient Selection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review