CoChat
Sign Up
Cited by 0Machine Learning in HealthcareBiomedical Text Mining and OntologiesElectronic Health Records Systems

TimeX: Phenotype Onset Extraction from Clinical Narratives

Open access
Chen, Fangyi, Jiang, Shiyi, Nguyen, Quan M., Ta, Casey N., Wang, Kai, Chung, Wendy K., Weng, Chunhua·2026·npj Health Systems
DOI 10.1038/s44401-026-00099-8PMC PMC13282170
Open this paper in CoChat — free accountOpen PDF

Abstract

Disease phenotype onset is critical for timely and accurate diagnosis and clinical decision-making, yet it remains poorly characterized in the literature. Estimating phenotype onset using electronic health record (EHR) data holds promise but remains challenging. Researchers often resort to EHR documentation timestamps as proxies for phenotype onset, which can be inaccurate. Conventional natural language processing (NLP) approaches suffer from limited scalability and generalizability, and struggle to interpret implicit or vague temporal expressions. To address these gaps, we introduce TimeX, a novel open-source pipeline that leverages Llama-3.1, using instruction-based prompting for extracting phenotype onset from clinical narratives. TimeX employs a modular workflow comprising family history filtering, phenotype extraction, negation handling, and temporal information extraction to estimate phenotype onset. It yielded an average accuracy of 81.24% in timestamp extraction using 102 manually annotated clinical notes from the Columbia University Irving Medical Center, substantially outperforming all five baselines by at least 14.86%. A case study of four rare disease cohorts revealed that narrative-derived phenotype onset is more precise than that based on documentation timestamps. TimeX supports accurate and scalable phenotype onset extraction, with the potential to enable more precise disease trajectory characterization and timely disease diagnosis.

AI snapshot

Written from the paper’s full text.

TL;DR

TimeX is an open-source, modular Llama-3.1-based pipeline that extracts phenotype onset from clinical narratives by filtering family history, extracting phenotypes, handling negation, and extracting/normalizing temporal expressions. It achieved 81.24% accuracy on truncated notes and 84.07% on full-length notes for phenotype onset extraction, outperforming all five baselines by at least 14.86%, and a rare-disease case study found narrative-derived onsets often earlier than documentation timestamps.

Key findings

  • TimeX achieved 81.24% (95% CI: 78.49–83.87%) accuracy for phenotype onset extraction using truncated clinical notes and 84.07% (95% CI: 81.32–86.4%) using full-length notes; family history exclusion reached 97.06% accuracy (precision 1.0, recall/sensitivity 0.92, F1 0.96, specificity 1.0). It substantially outperformed all five baselines by at least 14.86%, while the Llama-3.1-based baseline achieved only 19.27% (95% CI: 16.8–21.64%) accuracy for phenotype onset timestamp extraction.
  • TimeX achieved 82.92% accuracy (precision 0.76, recall 0.74, F1 0.75) in extracting verbatim time expressions; explicit temporal expressions were extracted at 78.38% (95% CI: 64.8–91.89%) accuracy and implicit timestamps at 70.31% (95% CI: 64.19–75.98%). In rare disease cohorts, TimeX achieved 85.87% accuracy for temporal expression extraction and 83.39% (95% CI: 79.15–87.29%) for temporal expression normalization.
  • In four rare disease cohorts, approximately 75–85% of phenotype terms extracted from clinical narratives lacked temporal information; among phenotypes with temporal attributes, 45% had estimated onsets more than 1 year before documented onsets, with significant differences between narrative-derived and documentation-based onsets (p < 0.001). Some phenotypes, including arthralgia, chronic pulmonary obstruction, hypertension, stroke, proteinuria, and congestive heart failure, had narrative-derived onsets preceding documentation-based estimates by at least 4 years.
  • Runtime averaged 4.87 s per narrative note; temporal expression extraction and normalization took 4.38 s per note, family history filtering 0.47 s per note, and phenotype extraction 0.023 s per note.

Methods

  • Design: TimeX is a modular LLM-based pipeline comprising data preprocessing/family history filtering with Llama-3.1-8B-Instruct, phenotype extraction with PhenoBERT and BERT-based negation detection, temporal expression extraction and normalization with Llama-3.1-8B-Instruct, and phenotype onset estimation integrating structured condition start times, note documentation datetimes, and narrative-extracted timestamps; the earliest onset per phenotype per patient was used as the final estimate.
  • Data and sample: Data came from the CUIMC Observational Medical Outcomes Partnership database (1985–2025). Manual evaluation used 102 randomly sampled clinical notes with 357 distinct phenotypes. The rare disease case study included Behcet syndrome (197 patients, 1656 notes), PAH (9363 patients, 24,700 notes), Fabry disease (103 patients, 3763 notes), and Whipple’s disease (63 patients, 2371 notes); 30 notes per cohort (120 total, 113 distinct phenotypes) were sampled for rare disease TIE evaluation.
  • Analysis: Accuracy, precision, recall/sensitivity, F1, specificity, and 95% CIs were computed; inter-annotator agreement was assessed with Cohen’s κ (>0.8 for all evaluations). Onset comparisons used the Friedman test followed by Dunn’s post hoc pairwise test with multiple-comparison correction. Baselines included a CoT step-by-step LLM baseline, SUTime, Gemma2-27B, Mistral-7B, and GPT-OSS-20B.

Limitations

  • Stated in the paper: much phenotype onset information is implicitly stated or missing in clinical notes, creating uncertainty; the finding that narrative-derived onset times may be more precise than documentation timestamps was supported only on a limited subset of cases, and larger-scale evaluation with rigorously annotated ground truth is needed; further evaluation on downstream tasks such as disease prediction is needed; future work could incorporate laboratory results, encounter details, and medication records.
  • Stated in the paper: lightweight LLMs have difficulty interpreting lengthy, complex clinical narratives, and TimeX showed improved performance on notes shorter than 3000 characters but no additional gain for notes shorter than 2500 characters, indicating a limitation in capturing cross-sentence event–time relationships. Error analysis found contextual misinterpretation (75.9%, N = 88), arithmetic errors (18.1%, N = 21), and reasoning errors (6%, N = 7).
  • Evident from the text: the evaluation and case study rely on retrospective data from a single institution (CUIMC), with substantial missing self-reported race and ethnicity information in the cohorts; the rare disease cohorts were selected for diseases emerging or typically diagnosed after adulthood, so generalizability to other settings, populations, and diseases is not fully established.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s44401-026-00099-8

Cites

58 of 58 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

21 of 29 cited sources checked · 7 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    LLMs tend to underperform when processing lengthy clinical notes with relevant temporal cues scattered across multiple sentences.

    “This truncation strategy was designed to mitigate the potential lost-in-the-middle problem48, where LLMs tend to underperform when processing lengthy clinical notes with relevant temporal cues scattered across multiple sentences.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [48] Liu, N. F. et al. Lost in the Middle: How language models use long contexts. Trans. Assoc. Comput. Linguist. 12, 157–173 (2024). source

      The Abstract and §1 Introduction report that language models often perform worse when relevant information appears in the middle of a long input context. The paper tests multi-document question answering and key-value retrieval, not lengthy clinical notes or temporal cues scattered across sentences. Page numbers are unavailable in our copy. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
  • Premisecitation

    The trained BERT-based model from Aken et al. effectively distinguishes affirmed from negated clinical concepts.

    “Phenotype negation detection was performed using the trained BERT-based model from Aken et al.47, which effectively distinguishes affirmed from negated clinical concepts.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [47] van Aken, B. et al. Assertion detection in clinical notes: medical language models to the rescue? In Proc. Second Workshop on Natural Language Processing for Medical Conversations (eds Shivade, C. et al.) 35–40. https://doi.org/10.18653/v1/2021.nlpmc-1.5 (Association for Computational Linguistics, 2021). source
  • Findingown data

    TimeX substantially outperformed all five baselines by at least 14.86% in timestamp extraction.

    “It yielded an average accuracy of 81.24% in timestamp extraction using 102 manually annotated clinical notes from the Columbia University Irving Medical Center, substantially outperforming all five baselines by at least 14.86%.”
    Quote in the paperSubstantial claim
  • Findingown data

    Among phenotype terms with temporal attributes, times were mainly expressed implicitly.

    “Among terms with temporal attributes, times were mainly expressed implicitly, such as “2 weeks ago” and “several months ago.””
    Quote in the paperSubstantial claim
  • Findingown data

    TimeX made a small number of reasoning errors where it inferred the wrong reference time.

    “Finally, we observed a small number of reasoning errors, where TimeX inferred the wrong reference time.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 14
  • Classified as probably not a substantial claim. 17
  • The quote only partly states this claim. 1

  • May combine two claims.

    Premisecitation

    HPO incorporates information on phenotype onset from existing literature, but only 1.2% of the phenotypic abnormality annotations had onset information.

    “only 1.2% of the phenotypic abnormality annotations had onset information36”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [36] Gargano, MA., Matentzoglu, N. & Coleman, B. et al. The Human Phenotype Ontology in 2024: phenotypes aroundthe world. Nucleic Acids Res 52, D1333–D1346, https://doi.org/10.1093/nar/gkad1005 (2024). source

      The ‘Updates to the annotations’ section reports new HPO annotations and their literature citations, but gives no percentage with onset information. The ‘GA4GH phenopacket schema’ section says phenotypic observations can specify onset; it does not establish that 1.2% of HPO phenotypic-abnormality annotations did so. Searches for ‘1.2%’ and onset information found no supporting passage in our copy. (LLM, conf=0.76)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    Conventional NLP approaches for temporal information extraction suffer from limited scalability and generalizability and struggle to interpret implicit or vague temporal expressions.

    “Conventional natural language processing (NLP) approaches suffer from limited scalability and generalizability, and struggle to interpret implicit or vague temporal expressions.”
    Quote in the paperSubstantial claim8 of 10 sources checked
    • cites [15] Tang, B. et al. A hybrid system for temporal information extraction from clinical text. J. Am. Med. Inform. Assoc. 20, 828–835 (2013). source

      The Background (page unavailable) reports that a temporal-expression toolkit developed for newswire performed poorly on clinical notes, supporting a domain-generalizability concern. The Discussion (page unavailable) says the authors’ approach to implicit date/time inference was very limited. The paper does not establish limited scalability or a general inability of conventional NLP approaches to interpret vague expressions. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [16] Xu, Y., Wang, Y., Liu, T., Tsujii, J. & Chang, E. I. An end-to-end system to identify temporal relation in discharge summaries: 2012 i2b2 challenge. J. Am. Med. Inform. Assoc. 20, 849–858 (2013). source

      The Introduction describes ambiguity in identifying temporal expressions and says some values require contextual inference. The Discussion also reports that the authors’ normalization system fails in some cases involving implicit time changes. These passages support the difficulty with implicit or ambiguous expressions, but do not establish the broader claim that conventional NLP approaches have limited scalability or generalizability. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites [17] D’Souza, J. & Ng, V. Classifying temporal relations in clinical data: a hybrid, knowledge-rich approach. J. Biomed. Inform. 46, S29–S39 (2013). source

      The Introduction says existing temporal-relation approaches typically relied on morpho-syntactic features and only a few semantic features. Methods §3.3 says implicit discourse relations are harder to identify than explicit ones. Neither passage establishes limited scalability or generalizability, or difficulty interpreting vague temporal expressions; the paper’s mention of a “Vague” relation class is not the same claim. No page numbers are available in our copy. (LLM, conf=0.78)

      Experimental — accuracy not yet measured
    • cites [18] Roberts, K., Rink, B. & Harabagiu, S. M. A flexible framework for recognizing events, temporal expressions, and temporal relations in clinical text. J. Am. Med. Inform. Assoc. 20, 867–875 (2013). source

      The Background (page not available) says temporal information can be ambiguous and that extraction methods need to be robust across hospitals, doctors, and record systems. The Results (page not available) identify normalization as the hardest temporal-expression task because it often requires discourse knowledge. These passages support difficulty with ambiguous or implicit timing and a concern about generalizability, but they do not establish that conventional NLP approaches generally have limited scalability. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [19] Sohn, S. et al. Comprehensive temporal information detection from clinical text: medical events, time, and TLINK identification. J. Am. Med. Inform. Assoc. 20, 836–842 (2013). source

      The Discussion (page unavailable) says relative time expressions requiring inference or reasoning were often difficult to catch, supporting the claim about implicit temporal expressions. It discusses difficulties with the authors’ system, however, and does not establish that conventional NLP approaches generally have limited scalability or generalizability. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [20] Lee, J. et al. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36, 1234–1240 (2020). source

      The Introduction says that NLP models trained on general-domain text have uncertain performance on biomedical text, offering limited support for the generalizability point. It does not say that conventional approaches have limited scalability or struggle with implicit or vague temporal expressions; a temporal-relation paper appears only in the References. No page numbers are available in our copy. (LLM, conf=0.77)

      Experimental — accuracy not yet measured
    • cites [21] Huang, K., Altosaar, J. & Ranganath, R. ClinicalBERT: modeling clinical notes and predicting hospital readmission. Preprint at arXiv https://doi.org/10.48550/arXiv.1904.05342 (2019). source
    • cites [22] Habibi, M., Weber, L., Neves, M., Wiegandt, D. L. & Leser, U. Deep learning with word embeddings improves biomedical named entity recognition. Bioinformatics 33, i37–i48 (2017). source

      The Introduction says conventional biomedical named-entity recognition tools are costly to develop and highly specialized, supporting a limited claim about their generalizability and ease of scaling. The paper does not address temporal information extraction or difficulty interpreting implicit or vague temporal expressions, so those parts of the attributed claim are unsupported. Page numbers are not supplied in our copy. (LLM, conf=0.87)

      Experimental — accuracy not yet measured
    • cites [23] Feng, Y., Qi, L. & Tian, W. PhenoBERT: a combined deep learning method for automated recognition of Human Phenotype Ontology. IEEE/ACM Trans. Comput. Biol. Bioinform. 20, 1269–1277 (2022). source
    • cites [24] Sun, C. et al. Biomedical named entity recognition using BERT in the machine reading comprehension framework. J. Biomed. Inform. 118, 103799 (2021). source

      The Introduction says traditional biomedical named-entity recognition relies on handcrafted, domain-specific features that are model- and entity-specific, which supports a limited-generalizability concern in that setting. It does not address scalability or the interpretation of implicit or vague temporal expressions in temporal information extraction; searches for those terms found no supporting passage. (LLM, conf=0.78)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    Temporal information in EHRs is often implicit, inconsistently documented, and embedded within unstructured narratives.

    “temporal information in EHRs is often implicit, inconsistently documented, and embedded within unstructured narratives8.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [8] Estiri, H., Strasser, Z. H. & Murphy, S. N. High-throughput phenotyping with temporal sequences. J. Am. Med. Inform. Assoc. 28, 772–781 (2021). source

      The Introduction says EHR observations are sampled irregularly and that a diagnosis timestamp may not reflect disease onset. The Background says precise timestamps are often unavailable for discrete clinical data. These passages support difficulty recovering temporal information, but they do not say that temporal information is embedded in unstructured narratives; the Background mentions clinical notes only as a source of keywords. (LLM, conf=0.86)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Premisecitation

    PhenoBERT is a state-of-the-art transformer-based model designed for phenotype extraction and normalization to HPO.

    “For this task, we adopted PhenoBERT23, a state-of-the-art transformer-based model designed for phenotype extraction and normalization to HPO35.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [23] Feng, Y., Qi, L. & Tian, W. PhenoBERT: a combined deep learning method for automated recognition of Human Phenotype Ontology. IEEE/ACM Trans. Comput. Biol. Bioinform. 20, 1269–1277 (2022). source
    • cites [35] Gargano, M. A. et al. The Human Phenotype Ontology in 2024: phenotypes around the world. Nucleic Acids Res 52, D1333–D1346 (2024). source

  • May combine two claims.

    Premisecitation

    Documentation timestamps reflect when events are recorded rather than when they occur, introducing systematic errors that can distort event sequencing, bias time-to-event analyses, and undermine model validity.

    “Documentation timestamps are often used as surrogates for disease onset; however, they reflect when events are recorded rather than when they occur, introducing systematic errors that can distort event sequencing, bias time-to-event analyses, and undermine model validity9–12.”
    Quote in the paperSubstantial claim3 of 4 sources checked
    • cites [9] Hou, J., Chan, S. F., Wang, X. & Cai, T. Risk prediction with imperfect survival outcome information from electronic health records. Biometrics 79, 190–202 (2023). source

      The Introduction (page not available) says that EHR diagnostic-code and note timings are imperfect proxies for true event time, and that using proxy times in risk models can cause substantial bias. It does not specifically establish that documentation timestamps systematically distort event sequencing or undermine model validity beyond that bias. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites [10] Ford, E. et al. What evidence is there for a delay in diagnostic coding of RA in UK general practice records? An observational study of free text. BMJ Open 6, e010393 (2016). source

      The Results (DMARD in text as a proxy for RA diagnosis) and Conclusions show that RA diagnostic codes can be entered well after earlier evidence of disease, so a code’s date may not mark disease onset. The paper discusses implications for case ascertainment and incidence estimates, but does not establish that this delay distorts event sequencing, biases time-to-event analyses, or undermines model validity. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites [11] Schulz, W. L. et al. Temporal relationship of computed and structured diagnoses in electronic health record data. BMC Med. Inform. Decis. Mak. 21, 61 (2021). source

      The Results (Temporality of clinical and computed diagnoses; page unavailable) show that structured diagnoses were recorded well after diagnoses could be computed from other EHR data. The Discussion (page unavailable) says these delays may bias cohort development, outcome ascertainment, and historical comparisons. The paper does not establish that documentation timestamps generally distort event sequencing, bias time-to-event analyses, or undermine model validity. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [12] Gai, X., Jiang, S. & Zhang, A. R. Subtype-aware registration of longitudinal electronic health records. J. Am. Stat. Assoc. 1, 21 (2026). source

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    TimeX was compared against a CoT step-by-step baseline adapted from recent temporal question-answering work.

    “We compared TimeX against a CoT step-by-step baseline, adapted from recent temporal question-answering work that evaluates temporal reasoning using the “when”-type template55.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [55] Kenneweg, S., Deigmöller, J., Cimiano, P. et al. TRAVELER: A Benchmark for Evaluating Temporal Reasoning Across Vague, Implicit and Explicit References. SN COMPUT. SCI. 7, 379 https://doi.org/10.1007/s42979-026-04973-y (2025). source
    • Rests on an assumption: Temporal information is often embedded within the free-text narratives.

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    TimeX was compared with a standard benchmark for TIE and normalization (SUTime).

    “Additionally, we compared TimeX with a standard benchmark for TIE and normalization (SUTime)56.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [56] Chang, Angel X., and Christopher D. Manning. Sutime: A library for recognizing and normalizing time expressions. Lrec. Vol. 12 (2012). source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Approaches using state-of-the-art open-source LLMs (Gemma2-27B, Mistral-7B, and GPT-OSS-20B) were used as baselines.

    “Besides, approaches using state-of-the-art open-source LLMs (Gemma2-27B57, Mistral-7B58, and GPT-OSS-20B59) were used as baselines as well.”
    Quote in the paperNot a substantial claim3 of 3 sources checked
    • cites [57] Riviere, M. et al. Gemma 2: Improving open language models at a practical size. Preprint at arXiv https://doi.org/10.48550/arXiv.2408.00118 (2024). source
    • cites [58] Jiang, A. Q. Mistral 7B. Preprint at arXiv https://doi.org/10.48550/arXiv.2310.06825 (2023). source
    • cites [59] Agarwal, S. et al. gpt-oss-120b & gpt-oss-20b Model Card. Preprint at arXiv https://doi.org/10.48550/arXiv.2508.10925 (2025). source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    TimeX occasionally made arithmetic mistakes when calculating exact dates.

    “TimeX also occasionally made arithmetic mistakes when calculating exact dates”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    TimeX's temporal expression extraction and normalization component required 4.38 s per note.

    “most of the computational time was concentrated in the temporal expression extraction and normalization component (4.38 s per note)”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

    Findingown data

    A total of 357 distinct phenotypes were identified from the 102 sampled clinical notes.

    “A total of 357 distinct phenotypes were identified from these notes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    TimeX offers a robust foundation for temporal phenotyping for diverse diseases, enabling more accurate disease characterization, better understanding of disease trajectories, earlier diagnosis, and improved patient care.

    “Besides its demonstrated performance and portability, TimeX offers a robust foundation for temporal phenotyping for diverse diseases, enabling more accurate disease characterization, better understanding of disease trajectories, earlier diagnosis, and improved patient care.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The segmentation-based approach effectively reduces irrelevant information and enhances model focus.

    “suggesting that the segmentation-based approach effectively reduces irrelevant information and enhances model focus.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    TimeX effectively retained only the patient's medical conditions by removing family history, achieving precise phenotype extraction.

    “It effectively retained only the patient’s medical conditions by removing family history, achieving precise phenotype extraction.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    TimeX is the first open-source, portable, end-to-end system specifically designed for temporal phenotype onset extraction.

    “To the best of our knowledge, TimeX is the first open-source, portable, end-to-end system specifically designed for temporal phenotype onset extraction.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The evaluation was performed on 102 randomly sampled clinical notes from the CUIMC database.

    “The evaluation was performed on 102 randomly sampled clinical notes from the CUIMC database.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    PhenoBERT demonstrated strong performance in identifying phenotype mentions across diverse clinical narratives while maintaining computational efficiency.

    “PhenoBERT demonstrated strong performance in identifying phenotype mentions across diverse clinical narratives while maintaining computational efficiency23.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [23] Feng, Y., Qi, L. & Tian, W. PhenoBERT: a combined deep learning method for automated recognition of Human Phenotype Ontology. IEEE/ACM Trans. Comput. Biol. Bioinform. 20, 1269–1277 (2022). source

  • May combine two claims.

    Interpretationreasoning

    TimeX has the potential to enable more precise disease trajectory characterization and timely disease diagnosis.

    “with the potential to enable more precise disease trajectory characterization and timely disease diagnosis.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    TimeX supports accurate and scalable phenotype onset extraction.

    “TimeX supports accurate and scalable phenotype onset extraction”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    TimeX establishes an important foundation for automated temporal phenotype extraction and demonstrates the feasibility of leveraging LLMs to capture complex temporal patterns in clinical narratives.

    “While further refinement is needed, this study establishes an important foundation for automated temporal phenotype extraction and demonstrates the feasibility of leveraging LLMs to capture complex temporal patterns in clinical narratives.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Incorporating temporal information from unstructured clinical narratives can improve onset estimation and potentially facilitate accurate modeling of disease progression.

    “Incorporating temporal information from unstructured clinical narratives can thus improve onset estimation and potentially facilitate accurate modeling of disease progression.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Narrative-derived phenotype onset is a more accurate approximation of true phenotype onset than documentation timestamps.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Sentences describing family history were removed from the raw clinical notes to avoid confounding phenotype extraction with hereditary context.

    “To avoid confounding phenotype extraction with hereditary context, sentences describing family history were removed from the raw clinical notes (e.g., “father with asthma and myocardial infarction”) using the Llama-3.1-8B-Instruct model46.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Cohen's κ was computed to assess the agreement between the two annotators.

    “Cohen’s κ was computed to assess the agreement between the two annotators, presented in Supplementary Table 7.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    An error analysis was conducted to categorize common failure patterns.

    “An error analysis was conducted to categorize common failure patterns, as described in the “Results” section.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    The Friedman test followed by Dunn's post hoc pairwise test with multiple-comparison correction was conducted to assess temporal discrepancies among onset estimates from documentation-based, narrative-based, and aggregated sources.

    “To assess the temporal discrepancies and identify significant differences among the onset estimates from documentation-based, narrative-based, and aggregated sources, we conducted the Friedman test followed by Dunn’s post hoc pairwise test with multiple-comparison correction53,54.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [53] Friedman, M. The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J. Am. Stat. Assoc. 32, 675–701 (1937). source
    • cites [54] Dunn, O. J. Multiple comparisons among means. J. Am. Stat. Assoc. 56, 52–64 (1961). source

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    Quality control measures excluded timestamps occurring before a patient's birthdate or after the corresponding note documentation time, as well as outliers with values in age greater than three times the standard deviation.

    “We applied quality control measures that excluded timestamps occurring before a patient’s birthdate or after the corresponding note documentation time, as well as outliers with values in age greater than three times the standard deviation.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    Two researchers independently reviewed all notes and assessed the precision of outputs yielded from the two variations of TimeX.

    “Two researchers (SJ and FC) each independently reviewed all notes and assessed the precision of outputs yielded from the two variations of TimeX, following the criteria defined in Table 3.”
    Quote in the paperSubstantial claim
27 of 27

  • The paper relies on this without support.

    Narrative-derived phenotype onset is a more accurate approximation of true phenotype onset than documentation timestamps.

    “Instead of relying solely on recorded timestamps, we hypothesize that temporal information extracted from narrative notes may provide a more accurate representation of the true onset and progression of patient symptoms and clinical events.”

  • The paper relies on this without support.

    Using LLMs for temporal information extraction from clinical text has not received much attention.

    “However, using LLMs for TIE has not received much attention.”

  • The paper relies on this without support.

    Individuals with rare diseases typically experience an average of 5 years between symptom onset and confirmed diagnosis.

    “Individuals with rare diseases often experience prolonged diagnostic delays, typically with an average of 5 years between symptom onset and confirmed diagnosis, and a substantial proportion of patients remain undiagnosed.”

  • The paper relies on this without support.

    The average diagnosis age derived from condition start timestamps is a valid proxy for the true average diagnosis age.

    “Age at diagnosis derived from condition start timestamps (Mean ± SD)”

  • The paper relies on this without support.

    The manually annotated clinical notes used for evaluation are representative of the full clinical note population.

    “using 102 manually annotated clinical notes from the Columbia University Irving Medical Center”

  • The paper relies on this without support.

    Much of the phenotype onset information is implicitly stated or missing in clinical notes, leading to uncertainty in estimating onset times.

    “Much of the phenotype onset information is implicitly stated or missing in clinical notes, leading to uncertainty in estimating onset times.”

  • The paper relies on this without support.

    Temporal expressions in clinical narratives often vary in both format and granularity.

    “Temporal expressions in clinical narratives often vary in both format and granularity; thus, we established a set of normalization rules to standardize them.”

  • The paper relies on this without support.

    Temporal information is often embedded within the free-text narratives.

    “Instead of relying on a series of structured events, this baseline directly uses clinical notes as input, reflecting a more realistic EHR context where temporal information is often embedded within the free-text narratives.”
8 of 8

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Experimental GPU hardware is reported inconsistently

    Major

    The figure note says all experiments used a single NVIDIA L40S GPU, while Methods says all experiments used a platform with four NVIDIA GeForce RTX 2080 Ti GPUs.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. Gargano, MA., Matentzoglu, N. & Coleman, B. et al. The Human Phenotype Ontology in 2024: phenotypes aroundthe world. Nucleic Acids Res 52, D1333–D1346, https://doi.org/10.1093/nar/gkad1005 (2024). · DOI · Paper page
  2. Paszke, A. et al. PyTorch: an imperative style, high-performance deep learning library. In Proc. 33rd International Conference on Neural Information Processing Systems (Curran Associates Inc., 2019).

Authors
Chen, F., Jiang, S., Nguyen, Q. M., Ta, C. N., Wang, K., Chung, W. K., & Weng, C.
Affiliations
  • Columbia University (US)
  • Children's Hospital of Philadelphia (US)
  • Boston Children's Hospital (US)
Corresponding author
Weng, Chunhua
ORCID
Venue
npj Health Systems
Publisher
Springer Science and Business Media LLC
Published
2026-06-19
Volume
3
Issue
1
Pages
49-49
ISSN
3005-1959 (ISSN-L)
DOI
10.1038/s44401-026-00099-8
PMC
PMC13282170
OpenAlex
W7165107768
Cited by
0
References
52
FWCI
0.00
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Health Systems (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • PubMed Central (submittedVersion, cc-by-nc-nd)
Funders
National Institutes of Health (R01HG013031, U01HG008680)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
Keywords
clinical text, clinical notes, temporal information extraction, Llama 3.1, negation handling, rare diseases, electronic health records, disease diagnosis, disease trajectories

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-27.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-27.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review