CoChat
Sign Up
Cited by 110Machine Learning in HealthcareArtificial Intelligence in Healthcare and EducationArtificial Intelligence in Healthcare

Learning the natural history of human disease with generative transformers

Open access
Shmatko, Artem, Jung, Alexander Wolfgang, Gaurav, Kumar, Brunak, Søren, Mortensen, Laust Hvas, Birney, Ewan, Fitzgerald, Tom, Gerstung, Moritz·2025·Nature
DOI 10.1038/s41586-025-09529-3PMC PMC12589094
Open this paper in CoChat — free account

Abstract

Decision-making in healthcare relies on understanding patients’ past and current health states to predict and, ultimately, change their future course 1–3 . Artificial intelligence (AI) methods promise to aid this task by learning patterns of disease progression from large corpora of health records 4,5 . However, their potential has not been fully investigated at scale. Here we modify the GPT 6 (generative pretrained transformer) architecture to model the progression and competing nature of human diseases. We train this model, Delphi-2M, on data from 0.4 million UK Biobank participants and validate it using external data from 1.9 million Danish individuals with no change in parameters. Delphi-2M predicts the rates of more than 1,000 diseases, conditional on each individual’s past disease history, with accuracy comparable to that of existing single-disease models. Delphi-2M’s generative nature also enables sampling of synthetic future health trajectories, providing meaningful estimates of potential disease burden for up to 20 years, and enabling the training of AI models that have never seen actual data. Explainable AI methods 7 provide insights into Delphi-2M’s predictions, revealing clusters of co-morbidities within and across disease chapters and their time-dependent consequences on future health, but also highlight biases learnt from training data. In summary, transformer-based models appear to be well suited for predictive and generative health-related tasks, are applicable to population-scale datasets and provide insights into temporal dependencies between disease events, potentially improving the understanding of personalized health risks and informing precision medicine approaches.

AI snapshot

Written from the paper’s full text.

TL;DR

Delphi-2M adapts GPT-2 to model continuous-time disease trajectories and predicts rates for more than 1,000 ICD-10 diagnoses conditional on past health history. Trained on UK Biobank and externally tested on Danish registries, it matches or exceeds many single-disease models and can generate synthetic future health trajectories.

Key findings

  • Delphi-2M was trained on 402,799 UK Biobank participants (80%) and validated on 100,639 (20%), with external testing on 1.93 million Danish individuals; it predicts 1,256 disease tokens plus death.
  • Internal validation average age-stratified AUC was approximately 0.76, with death at 0.97; performance decreased to an average of 0.70 after 10 years. External Danish testing gave average AUC 0.67 (s.d. 0.09) versus UK Biobank longitudinal testing 0.69 (s.d. 0.09), with disease-level correlation 0.76 (95% CI 0.72–0.80).
  • Generative sampling from age 60 reproduced future incidence at ages 70–75; 17% of disease tokens were correctly predicted in the first year, dropping to less than 14% after 20 years, compared with 12–13% using sex and age alone. A model trained only on synthetic data achieved average AUC 0.74, three percentage points lower than the original.
  • SHAP analyses showed digestive-tract diagnoses elevated pancreatic cancer risk 19-fold, and pancreatic cancer increased mortality rate almost ten thousandfold; cancer effects on mortality decayed over several years, whereas septicaemia effects were short-lived.

Methods

  • Design: modified GPT-2 architecture with continuous age encoding using sine/cosine functions, an added exponential waiting-time output head, and causal attention masks that also mask same-time tokens; trained with Adam for 200,000 iterations.
  • Data: UK Biobank first-occurrence ICD-10 level 3 diagnoses, cancer registry data, sex, BMI, smoking, alcohol, and death; Danish National Patient Registry and Causes of Death registry for external validation, restricted to ages 50–80 on 1 January 2016.
  • Analysis: age- and sex-stratified AUC, calibration, cross-entropy, longitudinal testing with a 1-year gap, generative trajectory sampling, UMAP of token embeddings, SHAP for token influence, and comparisons with clinical risk scores, biomarker models, and other machine learning methods.

Limitations

  • Stated: UK Biobank has healthy volunteer and participant selection biases, including immortal time bias before recruitment; missingness across self-report, primary care, hospital, cancer, and death sources is learned by the model; limited follow-up beyond age 80 makes that period unreliable; subgroup differences by ancestry and deprivation were observed; and temporal dependencies should not be interpreted as causal.
  • Stated: lifestyle token times were randomized by −20 to +40 years from recruitment to break confounding with mortality jumps, which “probably also diminishes the true effect of these tokens.”
  • Evident: external Danish validation showed lower average AUC (0.67) than UK Biobank longitudinal testing (0.69), and the paper notes that some learned associations may be artefacts of incomplete aggregated UK Biobank data rather than true diagnostic pathways.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41586-025-09529-3

Cites

54 of 54 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

2 of 2 cited sources checked · 0 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    UK Biobank participants tend to be on average more affluent and educated than the general population.

    “The UK Biobank comprises more white British citizens than the general population, and the participants tend to be on average more affluent and educated32.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [32] Fry, A. et al. Comparison of sociodemographic and health-related characteristics of UK Biobank participants with those of the general population. Am. J. Epidemiol. 186, 1026–1034 (2017). source

      The Discussion (page unavailable) says UK Biobank participants generally live in less socioeconomically deprived areas, supporting the claim’s affluence component as an area-level proxy. The paper passages found do not compare participants’ education with that of the general population, so the education component is unsupported. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
  • Premisecitation

    The UK Biobank comprises more white British citizens than the general population.

    “The UK Biobank comprises more white British citizens than the general population, and the participants tend to be on average more affluent and educated32.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [32] Fry, A. et al. Comparison of sociodemographic and health-related characteristics of UK Biobank participants with those of the general population. Am. J. Epidemiol. 186, 1026–1034 (2017). source

      Table 1 in the Results shows a higher proportion of white participants in UK Biobank (94.6%) than in the 2011 census population aged 40–69 (91.3%). But its footnote says the “White” category includes white British, white Irish, and other white backgrounds; the table does not establish a higher proportion specifically of white British people. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
  • Findingown data

    Death has an age-stratified AUC of 0.97 in both sexes.

    “Among the most confidently predicted next events is death, with an age-stratified AUC of 0.97 in both sexes.”
    Quote in the paperNot a substantial claim
  • Findingown data

    SHAP analysis reveals mutual dependencies by which each disease, sex and lifestyle token influences rate of subsequent disease tokens, similar to hazard ratios.

    “SHAP analysis of data from 100,639 individuals of the validation cohort reveals the mutual dependencies by which each disease, sex and lifestyle token influences the rate of subsequent disease tokens, similar to hazard ratios in conventional statistical models”
    Quote in the paperSubstantial claim
  • Findingown data

    A detailed analysis of UK Biobank's first occurrence data revealed several biases, reflected by Delphi-2M.

    “A detailed analysis of UK Biobank’s first occurrence data revealed several biases, reflected by Delphi-2M.”
    Quote in the paperSubstantial claim
  • May combine two claims. 20
  • Classified as probably not a substantial claim. 8

  • May combine two claims.

    Findingown data

    Self-reporting and GP records contain mostly common diseases, while hospital records include more-aggressive disease tokens such as myocardial infarction or septicaemia.

    “self-reporting and GP records contain mostly common diseases, while data from hospital records include more-aggressive disease tokens, such as myocardial infarction or septicaemia (Fig. 5d).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    When Delphi-2M is applied to Danish data, the average AUC is 0.67 (s.d. 0.09), lower than the 0.69 (s.d. 0.09) for longitudinal testing on UK Biobank data.

    “The average AUC when Delphi-2M is applied to Danish data was 0.67 (s.d. 0.09), which is lower than for longitudinal testing on UKB data (0.69, s.d. 0.09).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    Explainable AI methods reveal co-morbidity clusters within and across disease chapters and their time-dependent consequences, and highlight biases learnt from training data.

    “Explainable AI methods7 provide insights into Delphi-2M’s predictions, revealing clusters of co-morbidities within and across disease chapters and their time-dependent consequences on future health, but also highlight biases learnt from training data.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Delphi-2M reproduces the trends of lower disease rates among white ethnicity and higher diagnosis rates with deprivation.

    “Lower rates of diseases are reported among participants of white ethnicity, and the rate of diagnoses increases with deprivation—trends that are reproduced by Delphi-2M (Extended Data Fig. 9).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The effects of myocardial infarction or septicaemia regress within 5 years.

    “Delphi-2M’s ability to quantify the temporal influence of previous health data revealed that cancers increase mortality in a sustained manner, while the effects of myocardial infarction or septicaemia regress within 5 years.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The waiting times from the current event to each competing event are exponentially distributed with rates λi = exp(logitsi).
    • The authors state this as an assumption: The rates λi are constant in time over short periods.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    A version of the Charlson comorbidity index estimated with level 3 ICD-10 codes performed marginally worse than the version mapping level 3 codes to all possible level 4 codes.

    “We estimated a version with the level 3 codes as well and this did perform marginally worse.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    Delphi-2M's generative nature enables sampling synthetic future health trajectories, meaningful burden estimates up to 20 years, and training AI models without actual data.

    “Delphi-2M’s generative nature also enables sampling of synthetic future health trajectories, providing meaningful estimates of potential disease burden for up to 20 years, and enabling the training of AI models that have never seen actual data.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given

    Delphi's predictions are generally strongly influenced by statistical chance and compatible with a range of outcomes for a given individual.

    “Note that Delphi’s predictions are generally strongly influenced by statistical chance and compatible with a range of outcomes for a given individual.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In first year of sampling, 17% disease tokens are correctly predicted, dropping to less than 14% after 20 years, compared with 12-13% using sex and age alone.

    “In the first year of sampling, there are on average 17% disease tokens that are correctly predicted, and this drops to less than 14% 20 years later. These figures compare to values of 12–13% of correctly predicted disease tokens using sex and age alone”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    Internal longitudinal testing used data for all individuals still alive by the cut-off date (471,057) and evaluated incidence from 1 July 2021 to 1 July 2022, enforcing a 1 year data gap between predictions and evaluation.

    “Internal longitudinal testing was carried out using data for all individuals still alive by the cut-off date (471,057) and evaluated on incidence from 1 July 2021 to 1 July 2022, therefore enforcing a 1 year data gap between predictions and evaluation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Recruitment of UK Biobank participants between ages 40 and 70 creates a selection bias because no deaths are recorded before recruitment, which directly affects estimated mortality.

    “This creates a selection bias, as no deaths are recorded before recruitment, which has direct implications for the estimated mortality (Fig. 5b).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    In addition to a healthy volunteer bias and participant selection bias before recruitment, the diverse nature of health data sources impacted Delphi's prediction, as UK-Biobank-specific patterns of missingness were exploited to infer disease rates.

    “In addition to a healthy volunteer bias and participant selection bias before recruitment, the diverse nature of health data sources impacted Delphi’s prediction, as UK-Biobank-specific patterns of missingness were exploited to infer disease rates.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given

    Transformer-based models appear well suited for predictive and generative health tasks, applicable to population-scale datasets, and provide insights into temporal dependencies between disease events.

    “In summary, transformer-based models appear to be well suited for predictive and generative health-related tasks, are applicable to population-scale datasets and provide insights into temporal dependencies between disease events”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    Delphi-2M was trained on UK Biobank data and validated on external Danish data with no change in parameters.

    “We train this model, Delphi-2M, on data from 0.4 million UK Biobank participants and validate it using external data from 1.9 million Danish individuals with no change in parameters.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data

    External longitudinal testing on Danish registries included all individuals residing in Denmark 50-80 years of age on 1 January 2016, with predictions based on available data up to this point and evaluated on incidence from 1 January 2017 to 1 January 2018.

    “External longitudinal testing was conducted on the Danish registries. All individuals residing in Denmark 50–80 years of age on the 1 January 2016 were included. Predictions are based on the available data up to this point and were subsequently evaluated on incidence from 1 January 2017 to 1 of January 2018, similar to the internal longitudinal testing.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given

    Delphi-2M is uniquely capable of sampling future disease trajectories, which enables estimation of cumulative disease burdens over periods of up to 20 years, conditional on previous health information.

    “Delphi-2M is uniquely capable of sampling future disease trajectories, which enables the estimation of cumulative disease burdens over periods of up to 20 years, conditional on previous health information.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The waiting times from the current event to each competing event are exponentially distributed with rates λi = exp(logitsi).
    • The authors state this as an assumption: The rates λi are constant in time over short periods.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Under the null hypothesis of no between-subgroup AUC difference, the standardized residuals of subgroup AUCs follow a standard normal distribution.

    “Under the null hypothesis, these standardized residuals should follow a standard normal distribution.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Under the null hypothesis, the sum of squared standardized residuals across subgroups for a disease follows a chi-squared distribution with degrees of freedom equal to (n − 1), where n is the number of subgroups.

    “Under the null hypothesis, this sum follows a χ2 distribution with degrees of freedom equal to (n − 1), where n is the number of subgroups.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Predictions were evaluated on 0.96 million disease tokens across 796 ICD-10 codes, each with at least 25 cases.

    “Predictions were evaluated on 0.96 million disease tokens across 796 ICD-10 codes (each with at least 25 cases).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    Reproducibility of the method has been confirmed by retraining Delphi-2M using different train-validation splits (n = 4 independent experiments) and testing the trained model using longitudinal UK Biobank data and external data from the Danish National Patient Registry.

    “Reproducibility of the method has been confirmed by retraining Delphi-2M using different train-validation splits (n = 4 independent experiments; Supplementary Fig. 1) and testing the trained model using longitudinal UK Biobank data and external data from the Danish National Patient Registry.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductreasoning

    The longitudinal test design enforces an explicit cut-off and separates data so as to avoid any potential time-leakage.

    “This has two advantages: (1) we can enforce an explicit cut-off and separate data to avoid any potential time-leakage; and (2) we obtain insights into Delphi-2M prognostic capabilities and generalization.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Many patterns learned by Delphi-2M accurately reflect the true evolution of multi-morbidity, while there are differences within each cohort.

    “The fact that Delphi-2M can be applied to Danish population data with slightly reduced accuracy indicates that many patterns learned by the model accurately reflect the true evolution of multi-morbidity, while also highlighting the existence of differences within each cohort.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    Calibration analyses show predicted rates closely match observed cases, indicating rates of next tokens are consistently estimated.

    “calibration analyses in 5-year age brackets show that the predicted rates closely match the observed number of cases, showing that the models’ rates of the next tokens are consistently estimated”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Limited follow-up data are currently available for individuals 80 years of age and older, so this period is not reliably modelled by Delphi-2M.

    “Furthermore, limited follow-up data are currently available for individuals 80 years of age and older. This period is therefore not reliably modelled by Delphi-2M.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Under the null hypothesis that all population subgroups have the same true AUC (no bias), any observed between-subgroup differences in AUC would be attributable to statistical variance.

    “Under the null hypothesis that all subgroups have the same true AUC (no bias), any observed differences would be attributable to statistical variance.”
    Quote in the paperNot a substantial claim
25 of 25

  • The authors state this as an assumption.

    A person's health trajectory can be represented by a sequence of diagnoses using top-level ICD-10 codes recorded at the age of first diagnosis as well as death.

    “A person’s health trajectory can be represented by a sequence of diagnoses using top-level ICD-10 codes recorded at the age of first diagnosis as well as death.”

  • The authors state this as an assumption.

    Health cannot be understood only by the presentation of individual diagnoses but in the context of an individual's co-morbidities and their evolution over time.

    “Critically, health cannot only be understood by the presentation of individual diagnoses but, rather, in the context of an individual’s co-morbidities and their evolution over time.”

  • The authors state this as an assumption.

    The waiting times from the current event to each competing event are exponentially distributed with rates λi = exp(logitsi).

    “Assuming the Ti are each exponentially distributed waiting times with rates λi = exp(logitsi), the next event being j is equivalent to Tj being the first of the competing waiting times, that is, Tj = min Ti, or equivalently j = argmin Ti.”

  • The authors state this as an assumption.

    The rates λi are constant in time over short periods.

    “These approximations hold as long as the rates λi are constant in time, which is a reasonable assumption over short periods.”
4 of 4

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Different counts for the age-60 trajectory comparison

    Minor

    The main text says 63,662 sampled and observed trajectories were compared, while the Fig. 3 caption gives 63,622 individuals for the same experiment-a difference of 40.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. Lundberg, S. M. & Lee, S.-I. A unified approach to interpreting model predictions. Adv. Neural Inf. Process. Syst. 30, 4768–4777 (2017).
  2. Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. 30, 6000–6010 (2017).
  3. Steinberg, E., Fries, J. A., Xu, Y. & Shah, N. H. MOTOR: a time-to-event foundation model for structured medical records. Int. Conf. Learn. Represent. https://iclr.cc/virtual/2024/poster/18777 (2024)

Cited by

Sorted by citations

Authors
Shmatko, A., Jung, A. W., Gaurav, K., Brunak, S., Mortensen, L. H., Birney, E., Fitzgerald, T., & Gerstung, M.
Affiliations
  • European Bioinformatics Institute (GB)
  • University of Copenhagen (DK)
Corresponding author
  • Birney, Ewan
  • Gerstung, Moritz
ORCID
Venue
Nature
Publisher
Springer Science and Business Media LLC
Published
2025-09-17
Volume
647
Issue
8088
Pages
248-256
ISSN
  • 0028-0836 (ISSN-L)
  • 1476-4687
DOI
10.1038/s41586-025-09529-3
PMC
PMC12589094
OpenAlex
W4414274945
Semantic Scholar
d480376f7b315aba50e4b214b4bd545b6d63fefb
Cited by
110
References
42
Citations by year
2026: 92 · 2025: 18
FWCI
43.84
Type
journal article
Language
en
Update notices
correction (2025-11-12)
Open access
Licence
Locations
  • Nature (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Research at the University of Copenhagen (University of Copenhagen) (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (submittedVersion, cc-by)
Funders
Deutsches Krebsforschungszentrum
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
disease progression prediction, generative pre-trained transformer, UK Biobank, explainable AI, precision medicine
MeSH terms
Artificial Intelligence, Denmark, Disease, Female, Humans, Male, Comorbidity, Biological Specimen Banks, Disease Progression

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-27.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-27.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review