CoChat
Sign Up
Cited by 8Artificial Intelligence in Healthcare and EducationMachine Learning in HealthcareBiomedical Text Mining and Ontologies

Transforming oncology clinical trial matching through neuro-symbolic, multi-agent AI and an oncology-specific knowledge graph: a prospective evaluation in 3804 patients

Open access
Loaiza-Bonilla, A., Yost, C., Kurnaz, S., Tuysuz, E., Thaker, N.G., Giritlioglu, D., Noel Meza, J.P.·2026·ESMO Real World Data and Digital Oncology
DOI 10.1016/j.esmorw.2026.100706PMC PMC13091143
Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Clinical trial enrollment in oncology remains critically low, with fewer than 5% of eligible adults participating, in large part due to the complexity and labor intensity of eligibility screening. We prospectively evaluated a neuro-symbolic, multi-agent artificial intelligence (AI) platform integrating domain-specific large language model (LLM) agents, an oncology-specific knowledge graph, a real-time recommendation engine, and human-in-the-loop review to determine whether automated extraction and reasoning can safely improve trial identification, efficiency, and equity at scale. Methods: = 3804; Eastern Cooperative Oncology Group (ECOG) 0-2] balanced for cancer type incidence with metastatic or progressive malignancies were screened across a 12-month period. A multi-agent architecture-OncoAgents (LLM-based extraction and reasoning agents), OncoGraph (oncology knowledge graph), OncoRecommend (prioritization engine), and OncoSet (expert-curated corpus)-carried out automated data extraction, harmonization, and trial matching over 157 367 clinical pages (∼86.5 M tokens). Dual oncologists produced a gold standard of trial eligibility labels (Cohen's κ = 0.92). The primary unit of analysis was the patient-trial pair. Baselines included manual screening, GPT-4 zero-shot prompting, GPT-4 chain-of-thought, and frontier GPT-4o extraction/matching benchmarks. Outcomes included sensitivity, specificity, precision, F1 score, calibration of eligibility confidence scores, time-to-recommendation, fairness across demographic subgroups, and operational burden. Results: The multi-agent neuro-symbolic system achieved an F1 score of 0.82 (95% confidence interval 0.81-0.83). In comparison, the GPT-4 zero-shot baseline achieved an F1 of 0.47, and the GPT-4 chain-of-thought baseline achieved an F1 of 0.67. Per-patient screening time decreased from a median of 120 min (manual review) to ∼30 min total (15 min automated processing + 15 min clinical review). Across the cohort, the system processed 157 000 pages, screened 23 912 candidate patient-trial pairs, and produced 17 912 oncologist-confirmed matches, with median time-to-recommendation <7 days. No demographic subgroup exceeded a 10-percentage point F1 gap; the largest observed difference was ∼7 points between white and black/African American patients. Ablation experiments showed that both knowledge graph grounding and multi-agent decomposition contributed materially to performance and efficiency. Eligibility confidence scores exhibited reasonable calibration in the clinically relevant operating range. Conclusions: A neuro-symbolic, multi-agent architecture that couples LLM-based extraction with ontology-grounded, deterministic eligibility reasoning improved the accuracy, throughput, and timeliness of oncology clinical trial matching versus LLM-only baselines, while preserving clinician oversight and maintaining modest subgroup performance gaps. These results support scalable, equity-aware deployment of AI-assisted trial screening in routine oncology practice.

AI snapshot

Written from the paper’s full text.

TL;DR

A prospective evaluation of 3,804 patients found that a neuro-symbolic, multi-agent AI platform improved oncology clinical trial matching accuracy and efficiency versus LLM-only baselines while preserving human-in-the-loop review. The system achieved an F1 of 0.82 and reduced median per-patient screening time from ~120 minutes to ~30 minutes.

Key findings

  • The multi-agent neuro-symbolic system achieved F1 0.8246 (95% CI ~0.81–0.85), sensitivity 0.8375, specificity 0.8359, and precision 0.8121, compared with GPT-4 zero-shot F1 0.47, GPT-4 chain-of-thought F1 0.67, and GPT-4o matching accuracy 0.47.
  • Across 3,804 patients, the platform processed 157,367 clinical pages (~86.5M tokens), screened 23,912 candidate patient–trial pairs, and produced 17,912 oncologist-confirmed matches; median time-to-recommendation was <7 days (IQR 3–9 days).
  • Median per-patient screening time fell from ~120 minutes manually to ~30 minutes end-to-end with AI assistance (~15 minutes automated processing plus ~15 minutes clinician verification/discussion).
  • No demographic subgroup exceeded the pre-specified 10-percentage-point F1 gap; the largest observed difference was ~7 points between white and black/African American patients, associated with documentation and care fragmentation. Ablations reduced F1 from 0.8246 to 0.79 without OncoGraph grounding and to 0.78 with a single-agent variant.

Methods

  • Prospective evaluation of 3,804 consecutive patients with metastatic or progressive solid tumors (ECOG 0–2) across US academic–community sites over a 12-month period, with patients screened at referral and re-screened as new results accrued.
  • The platform combined OncoAgents (LLM extraction/reasoning agents), OncoGraph (oncology knowledge graph), OncoRecommend (prioritization engine), and OncoSet (expert-curated corpus); the primary unit of analysis was the patient–trial pair.
  • Dual board-certified oncologists produced a gold standard of trial eligibility labels (Cohen’s κ = 0.92); baselines included manual screening, GPT-4 zero-shot, GPT-4 chain-of-thought, and GPT-4o extraction/matching benchmarks, with outcomes including sensitivity, specificity, precision, F1, calibration, time-to-recommendation, fairness, and operational burden.

Limitations

  • The paper states that the evaluation reports identification efficiency—time, accuracy, calibration, and fairness—but not downstream endpoints such as consent, enrollment, or trial completion; accurate matching is necessary but not sufficient because logistical, socioeconomic, and consent barriers persist.
  • The paper states that the evaluation occurred within a single health system with relatively consistent documentation norms, and external validation across diverse EHR vendors, sites, and patient populations is essential before broad deployment.
  • The paper states that head-to-head comparison with every proprietary matching system was not feasible, and ablation analyses reflect the specific implementation; manual screening was a workflow reference, not a statistical F1 comparator.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.esmorw.2026.100706

Cites

16 of 16 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

7 of 9 cited sources checked · 1 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    Clinical trial enrollment in oncology is low, with fewer than 5% of eligible adults participating.

    “Clinical trial enrollment in oncology remains critically low, with fewer than 5% of eligible adults participating”
    • Also stated in: § The persistent crisis of oncology trial enrollment · plain
      “only 3%-5% of adult cancer patients enroll in research studies.1,2”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites [1] Monreal, 2025, Understanding the barriers to clinical trial referral and enrollment among oncology providers within the Veterans Health Administration, Mil Med source

      The Introduction describes cancer-trial participation as low, reporting that 2–8% of U.S. cancer patients participate. That range does not establish fewer than 5%, and it refers to cancer patients rather than eligible adults. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites [2] Eldridge, 2025, Barriers and opportunities for cancer clinical trials in low- and middle-income countries, JAMA Netw Open source

      The Introduction (page unavailable) says that few patients enroll in therapeutic cancer clinical trials, supporting the claim that enrollment is low. It does not give a rate for eligible adults or support the specific figure of fewer than 5%. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
  • Findingown data

    Extraction fidelity was high for internal EHR data (>95%).

    “Extraction fidelity was high for internal EHR data (>95%)”
    Quote in the paperSubstantial claim
  • Findingown data

    The GPT-4 chain-of-thought baseline achieved an F1 of 0.67.

    “the GPT-4 chain-of-thought baseline achieved an F1 of 0.67”
    • Also stated in: § Overall performance · plain
      “GPT-4 CoT baseline | 0.67 | 0.69 | 0.66 | 0.67”
    Quote in the paperSubstantial claim
  • Findingown data

    Extraction fidelity was modestly lower for external scanned referrals (∼89%).

    “and modestly lower for external scanned referrals (∼89%)”
    Quote in the paperNot a substantial claim
  • Findingown data

    The multi-agent neuro-symbolic system achieved an F1 score of 0.8246 (95% CI ~0.81-0.85) on the held-out test set.

    “The multi-agent neuro-symbolic system achieved an F1 score of 0.82 (95% confidence interval 0.81-0.83).”
    • Also stated in: § Overall performance · plain
      “The neuro-symbolic system achieved balanced sensitivity and specificity (∼0.84), precision 0.81, and F1 0.8246 (95% confidence interval ∼0.81-0.85).”
    • Also stated in: § Overall performance · plain
      “Multi-agent AI system | 0.8375 | 0.8359 | 0.8121 | 0.8246”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 21
  • May combine two claims. 26
  • Borderline: one of the tests for a substantial claim is unclear. 1

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    TRIALSCOPE demonstrates oncology-scale text normalization and ontology alignment.

    “Pipelines such as TRIALSCOPE demonstrate oncology-scale text normalization and ontology alignment.13”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [13] Habib, 2025, Computational framework for structuring and analyzing clinical trial criteria for AI-guided fine-grained matching, J Med Syst source

      The Methods section describes structuring eligibility criteria from three trial protocols, including one oncology trial. Searches for TRIALSCOPE and ontology alignment found no passage describing that pipeline or demonstrating oncology-scale text normalization and ontology alignment. The paper’s discussion of oncology criteria does not establish the attributed claim. (LLM, conf=0.76)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The system's features align with emerging trial protocol guidance (SPIRIT-AI; SPIRIT 2025).

    “These features align with emerging trial protocol guidance (SPIRIT-AI; SPIRIT 2025).15,16”
    Quote in the paperNot a substantial claim2 of 2 sources checked
    • cites [15] Rivera, 2020, Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension, BMJ source
    • cites [16] Chan, 2025, SPIRIT 2025 statement: updated guideline for protocols of randomized trials, BMJ source

      The Scope of SPIRIT 2025 section describes guidance on the minimum content of randomised-trial protocols, and the References section identifies SPIRIT-AI as an extension for protocols involving artificial intelligence (page numbers unavailable). The paper does not describe the citing system’s features, so it cannot establish that those features align with either guideline. (LLM, conf=0.76)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    Many published trial-matching approaches remain retrospective, disease-limited, or optimized for text similarity and retrieval rather than auditable patient-trial execution over longitudinal records.

    “many published approaches remain retrospective, disease-limited, or optimized for text similarity and retrieval rather than auditable patient–trial execution over longitudinal records.”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [3] Wornow, 2025, Zero-shot clinical trial patient matching with LLMs, NEJM AI source
    • cites [4] Jin, 2024, Matching patients to clinical trials with large language models, Nat Commun source

      The Introduction describes prior embedding-based similarity search as non-explainable, and the Discussion says this study did not evaluate longitudinal records or EHR integration. The paper’s evaluation uses existing datasets, but these passages do not establish that many published approaches are retrospective or disease-limited. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites [11] Kurnaz, 2024, Effect of a novel artificial intelligence (AI)–enabled multi-trial matching system on patient matching using real-world data, J Clin Oncol source

      The Methods (p. 1) describe a retrospective matching analysis of cancer patients, supporting this paper as an example of a retrospective, disease-specific approach. The paper does not assess whether many published approaches share those limitations, nor does it characterize approaches as optimized for text similarity or retrieval rather than auditable execution over longitudinal records. (LLM, conf=0.84)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    The estimated compute cost was low single-digit dollars per patient.

    “corresponding to an estimated low single-digit dollar compute cost”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Ablation experiments showed that both knowledge graph grounding and multi-agent decomposition contributed materially to performance and efficiency.

    “Ablation experiments showed that both knowledge graph grounding and multi-agent decomposition contributed materially to performance and efficiency.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Median model-processed volume was ∼23,000 tokens per patient.

    “Median model-processed volume was ∼23 000 tokens per patient”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The system processed approximately 86.5 million text tokens.

    “Approximately 86.5 million text tokens.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    Low-confidence or conflicting evidence (∼15% of patient-trial pairs) triggers prioritized human review.

    “Low-confidence or conflicting evidence (∼15% of patient–trial pairs) triggers prioritized human review.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The system processed 157,367 clinical document pages.

    “over 157 367 clinical pages (∼86.5 M tokens)”
    • Also stated in: § Data volume and processing · plain
      “157 367 clinical document pages.”
    Quote in the paperNot a substantial claim

  • Borderline on S3 (load-bearing): unclear from the text.

    Findingnone given

    Eligibility confidence scores exhibited reasonable calibration in the clinically relevant operating range.

    “Eligibility confidence scores exhibited reasonable calibration in the clinically relevant operating range.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Removing OncoGraph grounding reduced precision and F1 from 0.8246 to 0.79 and produced more hallucinated matches that violated numeric or temporal criteria.

    “Removing OncoGraph grounding (‘no-graph’ variant) reduced precision and F1 (0.8246→0.79) and produced more hallucinated matches that violated numeric or temporal criteria”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Eligibility confidence scores showed near-optimal calibration across typical screening thresholds, with mild underconfidence at lower bins and modest overconfidence at extremes.

    “Eligibility confidence scores showed near-optimal calibration across typical screening thresholds (Supplementary Table S5; Figure S1, available at https://doi.org/10.1016/j.esmorw.2026.100706), with mild underconfidence at lower bins and modest overconfidence at extremes.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The system outperformed zero-shot, CoT, and frontier GPT-4o models and reduced screening time fourfold.

    “By outperforming zero-shot, CoT, and frontier GPT-4o models and reducing screening time fourfold”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The neuro-symbolic multi-agent platform improved trial-matching accuracy, throughput, and timeliness while preserving clinician oversight and an equity-aware deployment framework in a prospective evaluation of 3804 consecutive patients.

    “a neuro-symbolic multi-agent platform improved trial-matching accuracy, throughput, and timeliness while preserving clinician oversight and an equity-aware deployment framework.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Common false-positive drivers included ambiguous note language, missing or inconsistently documented comorbidities, and unclear temporal relationships between therapies.

    “Common false-positive drivers included (Table 2):•Ambiguous note language (e.g. ‘history of pneumonitis’ without clarification of resolution or severity).•Missing or inconsistently documented comorbidities (e.g. remote myocardial infarction not reflected in structured problem lists).•Unclear temporal relationships between therapies (e.g. active versus prior immunotherapy lines).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Maintaining OncoGraph required ~20 h/month of expert curator time plus ~4 h/month of senior oncologist oversight.

    “We tracked maintenance effort to keep OncoGraph current: ∼20 h/month of expert curator time plus ∼4 h/month of senior oncologist oversight.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Disabling uncertainty-based triage increased reviewer time, whereas focusing review on the highest-uncertainty ~25% of pairs concentrated oversight where errors were most likely while preserving sensitivity.

    “Disabling uncertainty-based triage increased reviewer time by spreading attention across clear and ambiguous cases; focusing review on the highest-uncertainty ∼25% of pairs concentrated oversight where errors were most likely while preserving sensitivity.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Collapsing extraction and matching into a single LLM call lowered extraction fidelity and increased unresolved feature-criteria conflicts, with F1 falling to 0.78 versus 0.8246 for the full multi-agent system.

    “Collapsing extraction and matching into a single LLM call (‘single-agent’ variant) simplified the pipeline but lowered extraction fidelity and increased unresolved feature–criteria conflicts. F1 fell to 0.78 versus 0.8246 for the full multi-agent system”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Extraction/OCR artifacts accounted for 12% of AI error modes.

    “Extraction/OCR artifacts | 12% | Loss of fidelity in scanned PDF documents or faxed pathology reports.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    OncoSet served only for development and few-shot prompting; the prospective cohort was fully held out from prompt engineering and calibration.

    “OncoSet served only for development and few-shot prompting; the prospective cohort was fully held out from prompt engineering and calibration.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    All OncoGraph updates are version-controlled to support drift analysis.

    “All OncoGraph updates are version-controlled to support drift analysis”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The system ran in the health networks' secure environment under a HIPAA-aligned business associate agreement and patient consent.

    “The system ran in the health networks’ secure environment under a Health Insurance Portability and Accountability Act (HIPAA)-aligned business associate agreement and patient consent”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    A pre-specified absolute F1 gap >10 points between any two groups triggered a fairness alert.

    “A pre-specified absolute F1 gap >10 points between any two groups triggered a fairness alert.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    All LLM baselines were evaluated on the same filtered candidate trial sets.

    “all LLM baselines were evaluated on the same filtered candidate trial sets.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductreasoning

    Low-confidence extractions were routed for human verification, preventing upstream noise from propagating into false-positive matches and supporting robustness under documentation drift.

    “Low-confidence extractions were routed for human verification, preventing upstream noise from propagating into false-positive matches and supporting robustness under documentation drift.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    Eighty percent of cases were used for training and internal validation (10% split), with 20% held out for testing.

    “Eighty percent of cases were used for training and internal validation (10% split), with 20% held out for testing.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Improving fairness in this context is as much a documentation and interoperability problem as a model optimization problem.

    “improving fairness in this context is as much a documentation and interoperability problem as a model optimization problem.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The division of labor supports clinician review with concise evidence-linked rationales rather than opaque end-to-end prompts.

    “This division of labor supports clinician review with concise evidence-linked rationales rather than opaque end-to-end prompts.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Ongoing human review remains necessary as ontologies, biomarkers, and trial portfolios evolve.

    “ongoing human review remains necessary as ontologies, biomarkers, and trial portfolios evolve.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    Accurate matching is necessary but not sufficient for enrollment success.

    “accurate matching is necessary but not sufficient because logistical, socioeconomic, and consent barriers persist.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Logistical, socioeconomic, and consent barriers persist in trial enrollment.

  • May combine two claims.

    Premisecitation

    LLMs handle messy narrative understanding but are fragile on Boolean and temporal checks.

    “LLMs handle messy narrative understanding but are fragile on Boolean and temporal checks”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [10] Loaiza-Bonilla, 2025, Harnessing Moravec’s paradox in health care: a new era of collaborative intelligence, NEJM AI source

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Knowledge graph maintenance represents a form of intentional technical debt.

    “Knowledge graph maintenance therefore represents a form of intentional technical debt.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    Pure LLM pipelines often relocate the maintenance burden into repeated prompt tuning, silent failure modes, or post hoc chart review.

    “Pure LLM pipelines may appear lighter weight but often relocate this burden into repeated prompt tuning, silent failure modes, or post hoc chart review.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Distribution shift is likely to arise both from local documentation conventions and from evolving trial portfolios, biomarker requirements, and protocol language.

    “distribution shift is likely to arise both from local documentation conventions and from evolving trial portfolios, biomarker requirements, and protocol language.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Neuro-symbolic orchestration can be deployed prospectively at scale with transparent logic and measurable operational benefit.

    “a demonstration that neuro-symbolic orchestration can be deployed prospectively at scale with transparent logic and measurable operational benefit.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    OncoGraph contributes most strongly to precision by blocking logically inconsistent matches, whereas decomposition into extraction, normalization, and matching agents improves recall and reviewer efficiency.

    “OncoGraph appears to contribute most strongly to precision by blocking logically inconsistent matches, whereas decomposition into extraction, normalization, and matching agents improves recall and reviewer efficiency by reducing omitted clinical features and resolving conflicts across fragmented documentation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Both graph grounding and agentic decomposition materially contribute to performance and usability.

    “Together, these ablations indicate that both graph grounding and agentic decomposition materially contribute to performance and usability.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The fixed maintenance cost functions as a 'price of safety'-a deterministic logic layer that can be centrally audited and updated, unlike pure LLM systems that require continual re-prompting or retraining to correct hallucinations.

    “This fixed cost functions as a ‘price of safety’—a deterministic logic layer that can be centrally audited and updated, unlike pure LLM systems that require continual re-prompting or retraining to correct hallucinations.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given

    Our metrics align with or exceed published retrieval-augmented screening results in oncology, particularly in reducing hallucinated matches through knowledge graph grounding.

    “our metrics align with or exceed published retrieval-augmented screening results in oncology, particularly in reducing hallucinated matches through knowledge graph grounding.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Modular agents plus graph grounding can scale in routine practice, as shown in a cohort of 3804 patients and 23,912 patient-trial pairs.

    “showing that modular agents plus graph grounding can scale in routine practice (3804 patients; 23 912 patient-trial pairs).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Graph constraints enforce numeric and temporal rules, reduce hallucinated matches, and make reasoning auditable, while agent decomposition keeps extraction and matching focused and consistent.

    “graph constraints enforce numeric and temporal rules, reduce hallucinated matches, and make reasoning auditable, while agent decomposition keeps extraction and matching focused and consistent.”
    Quote in the paperSubstantial claim
41 of 41

  • The paper relies on this without support.

    Low enrollment is limited less by trial availability than by the effort of mapping fragmented longitudinal records to complex, exception-rich eligibility criteria.

    “Enrollment is limited less by trial availability than by the effort of mapping fragmented longitudinal records to complex, exception-rich eligibility criteria that are increasingly genomic, temporal, and organ-function specific.”

  • The paper relies on this without support.

    Manual screening typically takes ∼120 min per patient.

    “Manual screening typically takes ∼120 min per patient and requires multi-year chart review across disparate systems”

  • The paper relies on this without support.

    Fewer studies have prospectively evaluated broad oncology workflows combining multimodal EHR extraction, ontology normalization, deterministic temporal reasoning, and clinician review within one framework.

    “fewer studies have prospectively evaluated broad oncology workflows that combine multimodal electronic health record (EHR) extraction, ontology normalization, deterministic temporal reasoning, and clinician review within one framework.”

  • The paper relies on this without support.

    Even frontier LLMs (e.g. GPT-4o) remain probabilistic and can hallucinate criteria or mis-handle timing, making direct end-to-end deployment risky.

    “Even frontier LLMs (e.g. GPT-4o) remain probabilistic and can hallucinate criteria or mis-handle timing, making direct end-to-end deployment risky.”

  • The paper relies on this without support.

    Disparities in trial access and underrepresentation of black/African American patients in oncology trials are well documented.

    “given longstanding underrepresentation in oncology trials”
    “because disparities in trial access are already well documented”

  • The paper relies on this without support.

    Logistical, socioeconomic, and consent barriers persist in trial enrollment.

    “because logistical, socioeconomic, and consent barriers persist.”
6 of 6

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Conflicting confidence intervals for the system's F1 score

    Minor

    The abstract reports a 95% confidence interval of 0.81-0.83 for the multi-agent system's F1 score, while the Results section reports approximately 0.81-0.85 for the same score.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Vidal, 2025, Integrating knowledge graphs with symbolic AI: the path forward, Semant Web, 16, 123

Cited by

Sorted by citations

Authors
Loaiza-Bonilla, A., Yost, C., Kurnaz, S., Tuysuz, E., Thaker, N. G., Giritlioglu, D., & Noel Meza, J. P.
Affiliations
  • Creighton University (US)
  • Capital Health (US)
ORCID
Venue
ESMO Real World Data and Digital Oncology
Publisher
Elsevier BV
Published
2026-06
Volume
12
Pages
100706
ISSN
2949-8201 (ISSN-L)
DOI
10.1016/j.esmorw.2026.100706
PMC
PMC13091143
OpenAlex
W7151604493
Cited by
8
References
13
Citations by year
2026: 8
FWCI
16.17
Type
journal article
Language
en
Open access
Licence
Locations
  • ESMO Real World Data and Digital Oncology (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (submittedVersion, cc-by-nc-nd)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
Keywords
clinical trial recruitment, oncology, clinical trial eligibility screening, neuro-symbolic AI, multi-agent AI, large language models, clinical trial matching, automated data extraction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review