The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Evaluating eligibility criteria of oncology trials using real-world data and AI
Open accessAbstract
There is a growing focus on making clinical trials more inclusive but the design of trial eligibility criteria remains challenging1–3. Here we systematically evaluate the effect of different eligibility criteria on cancer trial populations and outcomes with real-world data using the computational framework of Trial Pathfinder. We apply Trial Pathfinder to emulate completed trials of advanced non-small-cell lung cancer using data from a nationwide database of electronic health records comprising 61,094 patients with advanced non-small-cell lung cancer. Our analyses reveal that many common criteria, including exclusions based on several laboratory values, had a minimal effect on the trial hazard ratios. When we used a data-driven approach to broaden restrictive criteria, the pool of eligible patients more than doubled on average and the hazard ratio of the overall survival decreased by an average of 0.05. This suggests that many patients who were not eligible under the original trial criteria could potentially benefit from the treatments. We further support our findings through analyses of other types of cancer and patient-safety data from diverse clinical trials. Our data-driven methodology for evaluating eligibility criteria can facilitate the design of more-inclusive trials while maintaining safeguards for patient safety. Trial Pathfinder uses data from electronic health records of patients with cancer to evaluate eligibility criteria and broaden restrictive criteria, which facilitates the design of more-inclusive trials while maintaining safeguards for patient safety.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Nivolumab versus docetaxel in advanced nonsquamous non-small-cell lung cancer
Borghaei, H., Paz-Ares, L., Horn, L., Spigel, D. R., Steins, M., Ready, N. E., Chow, L. Q., Vokes, E. E., Felip, E., Holgado, E., et al. (2015) · New England Journal of Medicine · doi:10.1056/nejmoa1507643 · Cited by 9,335 · cited 1×
“To investigate the generalizability of Trial Pathfinder to other types of cancer, we identified three additional trials in colorectal cancer, advanced melanoma and metastatic breast cancer with available trial protocols that can be encoded in the Flatiron database (Supplementary Table 32).”
Nivolumab versus Docetaxel in Advanced Squamous-Cell Non–Small-Cell Lung Cancer
Brahmer, J., Reckamp, K. L., Baas, P., Crinò, L., Eberhardt, W. E. E., Poddubskaya, E., Antonia, S., Pluzanski, A., Vokes, E. E., Holgado, E., et al. (2015) · New England Journal of Medicine · doi:10.1056/nejmoa1504627 · Cited by 8,447 · cited 1×
“We also applied Trial Pathfinder to 9,439 patients with aNSCLC who received Foundation Medicine genomic tests (Supplementary Tables 29–31).”
MethodPembrolizumab plus chemotherapy in metastatic non–small-cell lung cancer
Gandhi, L., Rodríguez-Abreu, D., Gadgeel, S., Esteban, E., Felip, E., De Angelis, F., Domine, M., Clingan, P., Hochmair, M. J., Powell, S. F., et al. (2018) · New England Journal of Medicine · doi:10.1056/nejmoa1801005 · Cited by 6,607
Herbst, R. S., Baas, P., Kim, D.-W., Felip, E., Pérez-Gracia, J. L., Han, J.-Y., Molina, J., Kim, J.-H., Arvis, C. D., Ahn, M.-J., et al. (2016) · The Lancet · doi:10.1016/s0140-6736(15)01281-7 · Cited by 6,449
Osimertinib in Untreated EGFR -Mutated Advanced Non–Small-Cell Lung Cancer
Soria, J.-C., Ohe, Y., Vansteenkiste, J., Reungwetwattana, T., Chewaskulyong, B., Lee, K. H., Dechaphunkul, A., Imamura, F., Nogami, N., Kurata, T., et al. (2018) · New England Journal of Medicine · doi:10.1056/nejmoa1713137 · Cited by 5,508
Rittmeyer, A., Barlesi, F., Waterkamp, D., Park, K., Ciardiello, F., von Pawel, J., Gadgeel, S. M., Hida, T., Kowalski, D. M., Dols, M. C., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(16)32517-x · Cited by 4,675
Pembrolizumab plus chemotherapy for squamous non-small-cell lung cancer
Paz-Ares, L., Luft, A., Vicente, D., Tafreshi, A., Gümüş, M., Mazières, J., Hermes, B., Çay Şenler, F., Csőszi, T., Fülöp, A., et al. (2018) · New England Journal of Medicine · doi:10.1056/nejmoa1810865 · Cited by 3,617 · cited 7×
“Relaxing restrictive eligibility criteria has the important benefit of making clinical trials more inclusive for diverse populations (Supplementary Tables 6–8).”
BackgroundInfluentialFrampton, G. M., Fichtenholtz, A., Otto, G. A., Wang, K., Downing, S. R., He, J., Schnall-Levin, M., White, J., Sanford, E. M., An, P., et al. (2013) · Nature Biotechnology · doi:10.1038/nbt.2696 · Cited by 2,157
Immortal Time Bias in Pharmacoepidemiology
Suissa, S. (2008) · American Journal of Epidemiology · doi:10.1093/aje/kwm324 · Cited by 1,708
Van Spall, H. G. C., Toren, A., Kiss, A., & Fowler, R. A. (2007) · JAMA · doi:10.1001/jama.297.11.1233 · Cited by 1,055
Singal, G., Miller, P. G., Agarwala, V., Li, G., Kaushik, G., Backenroth, D., Gossai, A., Frampton, G. M., Torres, A. Z., Lehnert, E. M., et al. (2019) · JAMA · doi:10.1001/jama.2019.3241 · Cited by 549
Kim, E. S., Bruinooge, S. S., Roberts, S., Ison, G., Lin, N. U., Gore, L., Uldrick, T. S., Lichtman, S. M., Roach, N., Beaver, J. A., et al. (2017) · Journal of Clinical Oncology · doi:10.1200/jco.2017.73.7916 · Cited by 485
Soria, J.-C., Felip, E., Cobo, M., Lu, S., Syrigos, K., Lee, K. H., Göker, E., Georgoulias, V., Li, W., Isla, D., et al. (2015) · The Lancet Oncology · doi:10.1016/s1470-2045(15)00006-6 · Cited by 433
Zhou, C., Wu, Y.-L., Chen, G., Liu, X., Zhu, Y., Lu, S., Feng, J., He, J., Han, B., Wang, J., et al. (2015) · Journal of Clinical Oncology · doi:10.1200/jco.2014.59.4424 · Cited by 372
Self-citationWu, Y.-L., Lu, S., Cheng, Y., Zhou, C., Wang, J., Mok, T., Zhang, L., Tu, H.-Y., Wu, L., Feng, J., et al. (2019) · Journal of Thoracic Oncology · doi:10.1016/j.jtho.2019.01.006 · Cited by 357
Ma, X., Long, L., Moon, S., Adamson, B. J. S., & Baxi, S. S. (2020) · medRxiv · doi:10.1101/2020.03.16.20037143 · Cited by 353
Huang, G. D., Bull, J., Johnston McKee, K., Mahon, E., Harper, B., & Roberts, J. N. (2018) · Contemporary Clinical Trials · doi:10.1016/j.cct.2018.01.003 · Cited by 294
Development and Validation of a High‐Quality Composite Real‐World Mortality Endpoint
Curtis, M. D., Griffith, S. D., Tucker, M., Taylor, M. D., Capra, W. B., Carrigan, G., Holzman, B., Torres, A. Z., You, P., Arnieri, B., et al. (2018) · Health Services Research · doi:10.1111/1475-6773.12872 · Cited by 217
Self-citationCriteria2Query: a natural language interface to clinical databases for cohort definition
Yuan, C., Ryan, P. B., Ta, C., Guo, Y., Li, Z., Hardin, J., Makadia, R., Jin, P., Shang, N., Kang, T., et al. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy178 · Cited by 170
Electronic medical records can be used to emulate target trials of sustained treatment strategies
Danaei, G., Rodríguez, L. A. G., Cantero, O. F., Logan, R. W., & Hernán, M. Á. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.11.021 · Cited by 152
Ni, Y., Wright, J., Perentesis, J., Lingren, T., Deleger, L., Kaiser, M., Kohane, I., & Solti, I. (2015) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-015-0149-3 · Cited by 142
Khozin, S., Miksad, R. A., Adami, J., Boyd, M. E., Brown, N. R., Gossai, A., Kaganman, I., Kuk, D., Rockland, J. M., Pazdur, R., et al. (2019) · Cancer · doi:10.1002/cncr.32383 · Cited by 139
Board on Health Care Services (2010) · National Academies Press eBooks · doi:10.17226/12879 · Cited by 136
EliIE: An open-source information extraction system for clinical trial eligibility criteria
Kang, T., Zhang, S., Tang, Y., Hruby, G. W., Rusanov, A., Elhadad, N., & Weng, C. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx019 · Cited by 136
An AI boost for clinical trials
Woo, M. (2019) · Nature · doi:10.1038/d41586-019-02871-3 · Cited by 134
Reducing patient eligibility criteria in cancer clinical trials.
George, S. L. (1996) · Journal of Clinical Oncology · doi:10.1200/jco.1996.14.4.1364 · Cited by 131
Modernizing Eligibility Criteria for Molecularly Driven Trials
Kim, E. S., Bernstein, D. S., Hilsenbeck, S. G., Chung, C. H., Dicker, A. P., Ersek, J. L., Stein, S., Khuri, F. R., Burgess, E. F., Hunt, K. K., et al. (2015) · Journal of Clinical Oncology · doi:10.1200/jco.2015.62.1854 · Cited by 115
Target trial emulation: teaching epidemiology and beyond
Labrecque, J. A., & Swanson, S. A. (2017) · European Journal of Epidemiology · doi:10.1007/s10654-017-0293-4 · Cited by 108
Miotto, R., & Weng, C. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocu050 · Cited by 104
Abernethy, A. P., Arunachalam, A., Burke, T. A., McKay, C., Cao, X., Sorg, R. A., & Carbone, D. P. (2017) · PLoS ONE · doi:10.1371/journal.pone.0178420 · Cited by 95
A Study in Contrasts: Eligibility Criteria in a Twenty-Year Sample of NSABP and POG Clinical Trials
Fuks, A., Weijer, C., Freedman, B., Shapiro, S. H., Skrutkowska, M., & Riaz, A. (1998) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(97)00240-0 · Cited by 78
Jonnalagadda, S., Adupa, A. K., Garg, R., Corona-Cox, J., & Shah, S. J. (2017) · Journal of Cardiovascular Translational Research · doi:10.1007/s12265-017-9752-2 · Cited by 66
Zhang, K., & Demner‐Fushman, D. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw176 · Cited by 62
Textual inference for eligibility criteria resolution in clinical trials
Shivade, C., Hebert, C. L., Lopetegui, M. A., de Marneffe, M.-C., Fosler‐Lussier, E., & Lai, A. M. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.09.008 · Cited by 46
Kim, J. H., Ta, C., Liu, C., Sung, C., Butler, A. M., Stewart, L. A., Ena, L., Rogers, J. R., Lee, J., Ostropolets, A., et al. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa276 · Cited by 29
Ni, Y., Beck, A. F., Taylor, R. G., Dyas, J. R., Solti, I., Grupp‐Phelan, J., & Dexheimer, J. W. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocv216 · Cited by 25
Sen, A. K., Ryan, P. B., Goldstein, A., Chakrabarti, S., Wang, S., Koski, E., & Weng, C. (2016) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.13195 · Cited by 25
Fehrenbacher, L., Ackerson, L., & Somkin, C. P. (2009) · Journal of Clinical Oncology · doi:10.1200/jco.2009.27.15_suppl.6538 · Cited by 21
Li, Q., He, Z., Guo, Y. M., Zhang, H., George, T. J., Hogan, W. R., Charness, N. H., & Bian, J. (2019) · PubMed · doi:10.48550/arxiv.1906.10163 · Cited by 12
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Artificial intelligence in drug development
Zhang, K., Yang, X., Wang, Y., Yu, Y., Huang, N., Li, G., Li, X., Wu, J. C., & Yang, S. (2025) · Nature Medicine · doi:10.1038/s41591-024-03434-4 · Cited by 498
Tan, Y. Y., Papez, V., Chang, W. H., Mueller, S. H., Denaxas, S., & Lai, A. G. (2022) · The Lancet Healthy Longevity · doi:10.1016/s2666-7568(22)00186-6 · Cited by 283
Transforming Cardiovascular Care With Artificial Intelligence: From Discovery to Practice
Khera, R., Oikonomou, E. K., Nadkarni, G. N., Morley, J. R., Wiens, J., Butte, A. J., & Topol, E. J. (2024) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2024.05.003 · Cited by 264
Big data in basic and translational cancer research
Jiang, P., Sinha, S., Aldape, K., Hannenhalli, S., Sahinalp, C., & Ruppin, E. (2022) · Nature Reviews Cancer · doi:10.1038/s41568-022-00502-0 · Cited by 254
Metadata
- Authors
- Liu, R., Rizzo, S., Whipple, S., Pal, N., Pineda, A. L., Lu, M., Arnieri, B., Lu, Y., Capra, W., Copping, R., & Zou, J.
- Venue
- Nature
- Published
- 2021-04-07
- Volume
- 592
- Issue
- 7855
- Pages
- 629-633
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W3140760072
- Cited by
- 322
- References
- 41
- FWCI
- 34.21
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.