The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index; 13 of the 15 sources its claims cite were checked for support.
Foresight-a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study
Open accessAbstract
BACKGROUND: An electronic health record (EHR) holds detailed longitudinal information about a patient's health status and general clinical history, a large portion of which is stored as unstructured, free text. Existing approaches to model a patient's trajectory focus mostly on structured data and a subset of single-domain outcomes. This study aims to evaluate the effectiveness of Foresight, a generative transformer in temporal modelling of patient data, integrating both free text and structured formats, to predict a diverse array of future medical outcomes, such as disorders, substances (eg, to do with medicines, allergies, or poisonings), procedures, and findings (eg, relating to observations, judgements, or assessments). METHODS: Foresight is a novel transformer-based pipeline that uses named entity recognition and linking tools to convert EHR document text into structured, coded concepts, followed by providing probabilistic forecasts for future medical events, such as disorders, substances, procedures, and findings. The Foresight pipeline has four main components: (1) CogStack (data retrieval and preprocessing); (2) the Medical Concept Annotation Toolkit (structuring of the free-text information from EHRs); (3) Foresight Core (deep-learning model for biomedical concept modelling); and (4) the Foresight web application. We processed the entire free-text portion from three different hospital datasets (King's College Hospital [KCH], South London and Maudsley [SLaM], and the US Medical Information Mart for Intensive Care III [MIMIC-III]), resulting in information from 811 336 patients and covering both physical and mental health institutions. We measured the performance of models using custom metrics derived from precision and recall. FINDINGS: Foresight achieved a precision@10 (ie, of 10 forecasted candidates, at least one is correct) of 0·68 (SD 0·0027) for the KCH dataset, 0·76 (0·0032) for the SLaM dataset, and 0·88 (0·0018) for the MIMIC-III dataset, for forecasting the next new disorder in a patient timeline. Foresight also achieved a precision@10 value of 0·80 (0·0013) for the KCH dataset, 0·81 (0·0026) for the SLaM dataset, and 0·91 (0·0011) for the MIMIC-III dataset, for forecasting the next new biomedical concept. In addition, Foresight was validated on 34 synthetic patient timelines by five clinicians and achieved a relevancy of 33 (97% [95% CI 91-100]) of 34 for the top forecasted candidate disorder. As a generative model, Foresight can forecast follow-on biomedical concepts for as many steps as required. INTERPRETATION: Foresight is a general-purpose model for biomedical concept modelling that can be used for real-world risk forecasting, virtual trials, and clinical research to study the progression of disorders, to simulate interventions and counterfactuals, and for educational purposes. FUNDING: National Health Service Artificial Intelligence Laboratory, National Institute for Health and Care Research Biomedical Research Centre, and Health Data Research UK.
Overview
CoChat Summary
TL;DR
Foresight is a GPT-2-based generative transformer that converts EHR free text into SNOMED concepts and forecasts future biomedical concepts across three hospital datasets. It achieved high precision@10 for next new disorders and concepts and clinician-rated relevancy on synthetic timelines, but the authors caution it reflects historical practice and is not ready for clinical decision support.
Key findings
- For forecasting the next new disorder, Foresight achieved precision@10 of 0.68 (SD 0.0027) for KCH, 0.76 (0.0032) for SLaM, and 0.88 (0.0018) for MIMIC-III; for the next new biomedical concept, precision@10 was 0.80 (0.0013), 0.81 (0.0026), and 0.91 (0.0011), respectively.
- In clinician validation on 34 synthetic patient timelines, the top forecasted candidate disorder was rated relevant in 33 of 34 cases (97% [95% CI 91–100]); overall interannotator agreement was 86%.
- Forecasting recurring concepts worked better than forecasting new concepts, and an LSTM-based approach performed 40% worse than the GPT-based model on next-concept prediction on the KCH dataset.
- MedCAT named entity recognition and linking achieved precision 0.9549 (95% CI 0.9519–0.9579), recall 0.8077 (0.8017–0.8137), and F1 0.8752 (0.8702–0.8802) with precision bias; without precision bias, precision was 0.9314 (0.9274–0.9354), recall 0.8959 (0.8909–0.9009), and F1 0.9133 (0.9093–0.9173).
Methods
- Retrospective modelling study using three EHR datasets: King’s College Hospital (KCH), South London and Maudsley (SLaM), and MIMIC-III; after preprocessing, the study covered 811,336 patients.
- The Foresight pipeline used CogStack for retrieval/preprocessing, MedCAT for named entity recognition and linking of free text to SNOMED concepts, Foresight Core (a GPT-2-based transformer for causal language modelling over biomedical concept tokens plus structured age, ethnicity, and sex), and a web application. Datasets were split randomly into 95% training and 5% test sets.
- Performance was measured with precision@N and recall@N (N=1, 5, or 10), where a forecast was considered correct if the concept appeared in a future time range of 30 days, 1 year, or infinity; outputs were filtered to match ground-truth concept type and new versus recurring status. Five clinicians generated 34 synthetic timelines and scored relevancy of forecasted disorders.
Limitations
- The authors state Foresight is derived from historical common practice, may not match contemporary recommended guidelines, and should not be used for clinical decision support; it prioritises probability over urgency and impact, rare diseases were removed for privacy, hallucinations are possible especially in long simulations, and the dataset does not fully encompass patient history and is weighted toward secondary/tertiary care.
- The authors state that considerably more tests are needed to validate long simulations forecasting many steps into the future.
- Evident from the text: evaluation is limited to three datasets—two UK NHS trusts and one US critical care dataset—so generalisability to other settings and populations is not established.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
11 Valid · 0 Not valid · 1 Not checked · 51 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“the disease profile would be weighted to conditions and scenarios in secondary and tertiary care-ie, it would be weighted towards more comorbidity because patients with lower complexity or earlystage conditions who are completely dealt with in primary care would be underrepresented in our dataset.23”
§ Foresight-a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study > Discussion
What the sources say
- Partly supportedcites [23] Bean, 2023, Hospital-wide natural language processing summarising the health data of 1 million patients, PLOS Digit Health sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“because about 80% of patient data are found in freetext format.1,2”
§ Foresight-a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study > Introduction
What the sources say
- Partly supportedcites [1] Jackson, 2018, CogStack—experiences of deploying integrated information retrieval and extraction services in a large National Health Service Foundation Trust hospital, BMC Med Inform Decis Mak sourceExperimental — accuracy not yet measured
The Background says that nearly all clinical information is recorded as unstructured clinical narrative, including free-text notes, discharge summaries, and referral letters. It does not give an 80% estimate, so it supports the general point but not the claimed figure. No page number is available in our copy. (LLM, conf=0.88)
- Could not verifycites [2] Hardy, 2022, Data consistency in the English Hospital Episodes Statistics database, BMJ Health Care Inform sourceExperimental — accuracy not yet measured
The Methods section (Study design and data collection; page unavailable) says HES data are taken from clinical notes and discharge summaries and entered by clinical coders. It gives no estimate of the proportion of patient data in free-text format. Searches for the 80% figure and for the share of unstructured patient records found no supporting passage. (LLM, conf=0.72)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The overall interannotator agreement was 86% (among the five clinicians).”
§ Foresight-a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Forecasting of recurring concepts worked much better than forecasting of new concepts (simply the case that someone with, for example, heart failure in their past timeline is likely to have it appear in their future timeline; table 2).”
§ Foresight-a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study > Results
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also tested a LSTMbased approach, but the perfor mance on the task of nextconcept prediction (for the category of all concept types) was 40% worse than the GPT based model on the KCH dataset (appendix p 6).”
§ Foresight-a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
8 more claims
Create a free account to see all 63 claims and every source verdict.
- Probably not a substantial claim. 18
- It may state two separate claims as one. 43
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Additionally, AI can predict how patients will respond to specific drugs based on their unique genetic makeup, facilitating the choice of the most suitable medication and dosage.94”
From A foundational architecture for AI agents in healthcare (2025) · cites this paper as [94]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Foresight-a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study > Discussion
“the F1 scores were 0•9280”
§ Foresight-a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study > Discussion
“(0•9240-0•8320) for experiencer”
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 1×
“…on the generated timelines with, for example, a learningtorank model similar to how the contrastive language–image pretraining (CLIP) model 21 works with DALLE. 22 Additionally, an intriguing future use case would be whether the knowledge representation of a historical EHR can be used as…”
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 1×
“Finally, hallucinations are also well described in transformerbased generative models, 24,25 including the recent ChatGPT, so such relevancy and mitigation systems would need to be built before the model would be suitable for clinical decision support.”
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y., Chen, D., Dai, W., et al. (2022) · arXiv · doi:10.48550/arxiv.2202.03629 · Cited by 4,981 · cited 1×
“Finally, hallucinations are also well described in transformerbased generative models, 24,25 including the recent ChatGPT, so such relevancy and mitigation systems would need to be built before the model would be suitable for clinical decision support.”
Zero-shot text-to-image generation
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., & Sutskever, I. (2021) · arXiv · doi:10.48550/arxiv.2102.12092 · Cited by 2,417 · cited 2×
“…timelines with, for example, a learningtorank model similar to how the contrastive language–image pretraining (CLIP) model 21 works with DALLE. 22 Additionally, an intriguing future use case would be whether the knowledge representation of a historical EHR can be used as a measure of…”
Rasmy, L., Xiang, Y., Xie, Z., Tao, C., & Zhi, D. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00455-y · Cited by 886
Vasey, B., Nagendran, M., Campbell, B., Clifton, D. A., Collins, G. S., Denaxas, S., Denniston, A. K., Faes, L., Geerts, B., Ibrahim, M., et al. (2022) · BMJ · doi:10.1136/bmj-2022-070904 · Cited by 665
Cited by
A Survey of Large Language Models in Medicine: Progress, Application, and Challenge
Zhou, H., Liu, F., Gu, B., Zou, X., Huang, J., Wu, J., Li, Y., Chen, S. S., Zhou, P., Liu, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.05112 · Cited by 240
“…generation or summarization has been proven to be more complete and more accurate than the human counterpart [139], there are still concerns with hallucination, as well as a tendency to approach inputs with a literal view instead of an assumption based perspective often taken by human doctors.”
BackgroundAssessing the research landscape and clinical utility of large language models: a scoping review
Park, Y.-J., Pillai, A., Deng, J., Guo, E., Gupta, M., Paget, M., & Naugler, C. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02459-6 · Cited by 187
AI chatbots not yet ready for clinical use.
Au Yeung, J., Kraljevic, Z., Luintel, A., Balston, A., Idowu, E., Dobson, R. J., & Teo, J. T. (2023) · Frontiers in Digital Health · doi:10.3389/fdgth.2023.1161098 · Cited by 176
Self-citationAI chatbots not yet ready for clinical use
Yeung, J. A., Kraljevic, Z., Luintel, A., Balston, A., Idowu, E., Dobson, R. J., & Teo, J. T. (2023) · doi:10.1101/2023.03.02.23286705 · Cited by 176
Self-citationFine-Tuning Large Language Models for Specialized Use Cases
Anisuzzaman, D., Malins, J. G., Friedman, P. A., & Attia, Z. I. (2024) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2024.11.005 · Cited by 264
Metadata
- Authors
- Kraljevic, Z., Bean, D., Shek, A., Bendayan, R., Hemingway, H., Yeung, J. A., Deng, A., Balston, A., Ross, J., Idowu, E., Teo, J. T., & Dobson, R. J. B.
- Affiliations
- National Health Service (GB)
- King's College London (GB)
- University College London Hospitals NHS Foundation Trust (GB)
- Guy's and St Thomas' NHS Foundation Trust (GB)
- Corresponding author
- Dobson, Richard J B
- ORCID
- Venue
- The Lancet Digital Health
- Publisher
- Elsevier BV
- Published
- 2024-04
- Volume
- 6
- Issue
- 4
- Pages
- e281-e290
- ISSN
- 2589-7500 (ISSN-L)
- PMC
- PMC11220626
- OpenAlex
- W4393021028
- Cited by
- 169
- References
- 28
- Citations by year
- 2026: 74 · 2025: 58 · 2024: 27
- FWCI
- 17.13
- Type
- journal article
- Language
- en
- Update notices
- erratum (2024-10-01)
- Open access
- gold
- doi.org
- Licence
- cc-by
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by/4.0/ (vor)
- Locations
- The Lancet Digital Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion, cc-by)
- Research Portal (King's College London) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- UKRI
- NIHR Maudsley BRC
- Health Data Research UK
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- forecasting, generative pre-trained transformer, electronic health records, named entity recognition, named entity linking, clinical text, temporal modeling, disorder prediction, MIMIC-III, King's College Hospital
- MeSH terms
- Artificial Intelligence, Humans, Mental Health, Retrospective Studies, State Medicine, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-27, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-27 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Foresight-a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study. https://app.cochat.ai/papers/10.1016/s2589-7500(24)00025-6
- The paper
- Kraljevic, Z., Bean, D., Shek, A., Bendayan, R., Hemingway, H., Yeung, J. A., Deng, A., Balston, A., Ross, J., Idowu, E., Teo, J. T., & Dobson, R. J. B. (2024). Foresight-a generative pretrained transformer for modelling of patient timelines using electronic health records: a retrospective modelling study. The Lancet Digital Health. https://doi.org/10.1016/s2589-7500(24)00025-6
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.