Classifying temporal relations in clinical data: A hybrid, knowledge-rich approach
Open accessAbstract
We address the TLINK track of the 2012 i2b2 challenge on temporal relations. Unlike other approaches to this task, we (1) employ sophisticated linguistic knowledge derived from semantic and discourse relations, rather than focus on morpho-syntactic knowledge; and (2) leverage a novel combination of rule-based and learning-based approaches, rather than rely solely on one or the other. Experiments show that our knowledge-rich, hybrid approach yields an F-score of 69.3, which is the best result reported to date on this dataset.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Conventional NLP approaches for temporal information extraction suffer from limited scalability and generalizability and struggle to interpret implicit or vague temporal expressions.
“Conventional natural language processing (NLP) approaches suffer from limited scalability and generalizability, and struggle to interpret implicit or vague temporal expressions.”
From TimeX: Phenotype Onset Extraction from Clinical Narratives (2026) · cites this paper as [17]
The Introduction says existing temporal-relation approaches typically relied on morpho-syntactic features and only a few semantic features. Methods §3.3 says implicit discourse relations are harder to identify than explicit ones. Neither passage establishes limited scalability or generalizability, or difficulty interpreting vague temporal expressions; the paper’s mention of a “Vague” relation class is not the same claim. No page numbers are available in our copy. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 22
- [1]Chambers N Wang S Jurafsky D 2007 Classifying temporal relations between events Proceedings of ACL Companion Volume: Proceedings of the Demo and Poster Sessions 173 176
- [2]Chambers N Jurafsky D 2008 Jointly combining implicit constraints improves temporal ordering Proceedings of EMNLP 698 706
- [3]Cherry C Zhu X Martin J de Bruijn B 2013 À la Recherche du Temps Perdu: extracting temporal relations from medical text in the 2012 i2b2 NLP challenge Journal of the American Medical Informatics Association PMC3756270 23523875
- [4]Chklovski T Pantel P 2004 Verbocean: Mining the web for fine-grained semantic verb relations Proceedings of EMNLP 33 40
- [5]Collobert R Weston J Bottou L Karlen M Kavukcuoglu K Kuksa PP 2011 Natural language processing (almost) from scratch Journal of Machine Learning Research 12 2493 2537
- [6]Finkel J Dingare S Manning C Nissim M Alex B Grover C 2005 Exploring the boundaries: gene and protein identification in biomedical text BMC Bioinformatics 6 S5 15960839 PMC1869019
- [7]Ha EY Baikadi A Licata C Lester J 2010 NCSU: Modeling temporal relations with markov logic and lexical ontology Proceedings of the 5th International Workshop on Semantic Evaluation 341 344
- [8]Hitzeman J Moens M Grover C 1995 Algorithms for analysing the temporal structure of discourse Proceedings of the 7th Conference of the EACL 253 260
- [10]Llorens H Saquete E Navarro B 2010 TIPSem (English and Spanish): Evaluating CRFs and semantic roles in TempEval-2 Proceedings of the 5th International Workshop on Semantic Evaluation 284 291
- [11]Mani I Verhagen M Wellner B Lee CM Pustejovsky J 2006 Machine learning of temporal relations Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics 753 760
- [12]McDonald R Pereira F 2005 Identifying gene and protein mentions in text using conditional random fields BMC Bioinformatics 6 S6 15960840 PMC1869020
- [13]Min C Srikanth M Fowler A 2007 LCC-TE: A hybrid approach to temporal relation identification in news text Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007) 219 222
- [14]Prasad R Dinesh N Lee A Miltsakaki E Robaldo L Joshi A Webber B 2008 The penn discourse tree-bank 2.0 Proceedings of the 6th International Conference on Language Resources and Evaluation
- [15]Puşcaşu G 2007 WVALI: Temporal relation identification by syntactico-semantic analysis Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007) 484 487
- [17]Sun W Rumshisky A Uzuner O 2013 Evaluating temporal relations in clinical text: 2012 i2b2 Challenge Journal of American Medical Informatics Assoication 10.1136/amiajnl-2013-001628 PMC3756273 23564629 · DOI
- [18]Tang B Wu Y Jiang M Chen Y Denny J Xu H 2012 Extracting temporal information from clinical text — Vanderbilt’s system for the 2012 i2b2 NLP challenge Proceedings of the 6th i2b2 Shared Tasks and Workshop on Challenges in Natural Langauge Processing for Clincial Data
- [19]Tsochantaridis I Hofmann T Joachims T Altun Y 2004 Support vector machine learning for interdependent and structured output spaces Proceedings of the 21st ICML 104 112
- [20]UzZaman N Allen JF 2010 TRIPS and TRIOS system for TempEval-2: Extracting temporal information from text Proceedings of the Fifth International Workshop on Semantic Evaluations (SemEval-2010) 276 283
- [21]Verhagen M Gaizauskas R Schilder F Hepple M Katz G Pustejovsky J 2007 SemEval-2007 Task 15: TempEval temporal relation identification Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007) 75 80
- [22]Verhagen M Sauri R Caselli T Pustejovsky J 2010 SemEval-2010 Task 13: TempEval-2 Proceedings of the 5th International Workshop on Semantic Evaluation 57 62
- [23]Xu Y Wang Y Liu T Tsujii J Chang E 2012 An end-to-end system to identify temporal relation in discharge summaries Proceedings of the 6th i2b2 Shared Tasks and Workshop on Challenges in Natural Langauge Processing for Clincial Data
- [24]Zhou G Shen D Zhang J Su J Tan S 2005 Recognition of protein/gene names from text using an ensemble of classifiers BMC Bioinformatics 6 S7 15960841 PMC1869021
Cited by
Sorted by citations
TimeX: Phenotype Onset Extraction from Clinical Narratives
Chen, F., Jiang, S., Nguyen, Q. M., Ta, C. N., Wang, K., Chung, W. K., & Weng, C. (2026) · npj Health Systems · doi:10.1038/s44401-026-00099-8
- Authors
- D’Souza, J., & Ng, V.
- Affiliations
- The University of Texas at Dallas (US)
- Corresponding author
- D’Souza, Jennifer
- Venue
- Journal of Biomedical Informatics
- Publisher
- Elsevier BV
- Published
- 2013-12
- Volume
- 46
- Pages
- S29-S39
- ISSN
- 1532-0464 (ISSN-L)
- PMC
- PMC3855590
- OpenAlex
- W2011959271
- Cited by
- 18
- References
- 0
- Citations by year
- 2026: 1 · 2024: 1 · 2023: 2 · 2021: 2 · 2020: 2 · 2019: 1 · 2017: 1 · 2016: 2 · 2015: 2 · 2014: 1 · 2013: 3
- FWCI
- 1.20
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Journal of Biomedical Informatics (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- NIH NLM (2U54LM008748 and 1R13LM011411-01)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- temporal relations, clinical text, semantic relations, discourse relations, clinical data annotation
- MeSH terms
- Humans, Medical Informatics, Natural Language Processing, Semantics, Time Factors, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
