Clustering clinical trials with similar eligibility criteria features
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Objectives To automatically identify and cluster clinical trials with similar eligibility features. Methods Using the public repository ClinicalTrials.gov as the data source, we extracted semantic features from the eligibility criteria text of all clinical trials and constructed a trial-feature matrix. We calculated the pairwise similarities for all clinical trials based on their eligibility features. For all trials, by selecting one trial as the center each time, we identified trials whose similarities to the central trial were greater than or equal to a predefined threshold and constructed center-based clusters. Then we identified unique trial sets with distinctive trial membership compositions from center-based clusters by disregarding their structural information. Results From the 145,745 clinical trials on ClinicalTrials.gov, we extracted 5,508,491 semantic features. Of these, 459,936 were unique and 160,951 were shared by at least one pair of trials. Crowdsourcing the cluster evaluation using Amazon Mechanical Turk (MTurk), we identified the optimal similarity threshold, 0.9. Using this threshold, we generated 8,806 center-based clusters. Evaluation of a sample of the clusters by MTurk resulted in a mean score 4.331±0.796 on a scale of 1–5 (5 indicating “strongly agree that the trials in the cluster are similar”). Conclusions We contribute an automated approach to clustering clinical trials with similar eligibility features. This approach can be potentially useful for investigating knowledge reuse patterns in clinical trial eligibility criteria designs and for improving clinical trial recruitment. We also contribute an effective crowdsourcing method for evaluating informatics interventions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172 · cited 3×
“To assure that workers were paying attention and not just randomly selecting answers, and to filter out ‘‘spammers’’ or ‘‘bots’’ [32], we inserted the hidden phrase ‘‘Waldo is hiding here’’ into one of the comparison text pairs.”
InfluentialCheap and fast---but is it good?: evaluating non-expert annotations for natural language tasks
Snow, R., O'Connor, B., Jurafsky, D., & Ng, A. Y. (2008) · Proceedings of the Conference on Empirical Methods in Natural Language Processing - EMNLP '08 · doi:10.3115/1613715.1613751 · Cited by 1,887
Campbell, M., Snowdon, C., Francis, D., Elbourne, D., McDonald, A., Knight, R., Entwistle, V, Garcia, J., Roberts, I., Grant, A., et al. (2007) · Health Technology Assessment · doi:10.3310/hta11480 · Cited by 433
Cited by
Health Natural Language Processing: Methodology Development and Applications
Hao, T., Huang, Z., Liang, L., Weng, H., & Tang, B. (2021) · JMIR Medical Informatics · doi:10.2196/23898 · Cited by 48
Self-citationIsojarvi, J., Wood, H., Lefebvre, C., & Glanville, J. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1294 · Cited by 38
The INTEGRATE project: delivering solutions for efficient multi-centric clinical research and trials
Kondylakis, H., Claerhout, B., Keyur, M., Koumakis, L., van Leeuwen, J., Marias, K., Perez-Rey, D., De Schepper, K., Tsiknakis, M., & Bucur, A. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.05.006 · Cited by 18
Embi, P. J., & Payne, P. R. O. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.10.007 · Cited by 13
Metadata
- Authors
- Hao, T., Rusanov, A., Boland, M. R., & Weng, C.
- Venue
- Journal of Biomedical Informatics
- Publisher
- Elsevier BV
- Published
- 2014-12
- Volume
- 52
- Pages
- 112-120
- ISSN
- 1532-0464 (ISSN-L)
- OpenAlex
- W2134689819
- Cited by
- 68
- References
- 3
- FWCI
- 3.65
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Clustering clinical trials with similar eligibility criteria features. https://app.cochat.ai/papers/10.1016/j.jbi.2014.01.009
- The paper
- Hao, T., Rusanov, A., Boland, M. R., & Weng, C. (2014). Clustering clinical trials with similar eligibility criteria features. Journal of Biomedical Informatics. https://doi.org/10.1016/j.jbi.2014.01.009
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.