CoChat
Sign Up
Cited by 39Health Systems, Economic Evaluations, Quality of LifeStatistical Methods in Clinical TrialsSimulation Techniques and Applications

An Introductory Tutorial on Cohort State-Transition Models in R Using a Cost-Effectiveness Analysis Example

Open access
Alarid-Escudero, Fernando, Krijkamp, Eline, Enns, Eva A., Yang, Alan, Hunink, M. G. Myriam, Pechlivanoglou, Petros, Jalal, Hawre·2023·Medical Decision Making
DOI 10.1177/0272989x221103163PMC PMC9742144
Open this paper in CoChat — free accountOpen PDF

Abstract

Decision models can combine information from different sources to simulate the long-term consequences of alternative strategies in the presence of uncertainty. A cohort state-transition model (cSTM) is a decision model commonly used in medical decision making to simulate the transitions of a hypothetical cohort among various health states over time. This tutorial focuses on time-independent cSTM, in which transition probabilities among health states remain constant over time. We implement time-independent cSTM in R, an open-source mathematical and statistical programming language. We illustrate time-independent cSTMs using a previously published decision model, calculate costs and effectiveness outcomes, and conduct a cost-effectiveness analysis of multiple strategies, including a probabilistic sensitivity analysis. We provide open-source code in R to facilitate wider adoption. In a second, more advanced tutorial, we illustrate time-dependent cSTMs.

AI snapshot

Written from the paper’s full text.

TL;DR

This tutorial conceptualizes time-independent cohort state-transition models (cSTMs) and demonstrates their implementation in base R using a 4-state Sick-Sicker cost-effectiveness analysis example. It provides open-source code and illustrates CEA, probabilistic sensitivity analysis, and related outputs such as ICERs, CEACs, CEAFs, and expected loss curves.

Key findings

  • The tutorial implements time-independent cSTMs in R without requiring a specialized cSTM package, using a previously published 4-state Sick-Sicker model to compare standard of care (SoC), strategy A, strategy B, and strategy AB.
  • In the base-case CEA, SoC is least costly and effective; strategy B yields an expected incremental benefit of 1.473 QALYs per individual for an additional expected cost of $107,521, with an ICER of $72,988/QALY; strategy AB has an ICER of $125,764/QALY; strategy A is dominated.
  • In the PSA with 1,000 parameter sets, SoC has the highest probability of being cost-effective and highest expected NMB at WTP thresholds below $80,000/QALY; strategy B is optimal between $80,000 and $120,000/QALY; strategy AB is optimal at WTP thresholds greater than or equal to $120,000/QALY.
  • The tutorial also demonstrates within-cycle correction using Simpson’s 1/3rd rule and discounting of costs and QALYs at an annual rate of 3%.

Methods

  • Tutorial design: step-by-step mathematical conceptualization and R implementation of time-independent cSTMs, with code in a GitHub repository; the case study is a previously published 4-state Sick-Sicker model for a CEA of four strategies (SoC, A, B, AB).
  • Cohort and model: hypothetical cohort of 25-year-olds starting in the “Healthy” state, simulated until maximum age 100 years using annual cycles (75 one-year cycles); transition rates are converted to probabilities, and costs and QALYs are discounted at 3% annually with within-cycle correction.
  • Analysis: base-case cost-effectiveness analysis with ICERs, plus probabilistic sensitivity analysis sampling 1,000 parameter sets from distributions (gamma for transition rates, lognormal for hazard ratios, beta for utility weights), producing cost-effectiveness scatter plots, CEACs, CEAFs, and expected loss curves.

Limitations

  • The paper states that the example assumes all parameters are known or that their uncertainty distributions are known; real-world cSTMs require thorough evidence synthesis, and the PSA assumes all parameters are independent, whereas parameters could be correlated or rank ordered.
  • The paper states that cSTMs are recommended when the number of states is “not too large,” that unit testing is beyond the tutorial’s scope, and that the tutorial focuses on time-independent cSTMs only, with time-dependent models covered in an advanced tutorial.
  • Evident from the text: the case study is a hypothetical disease and illustrative model, not a real-world clinical or policy evaluation, and the reported results are specific to the Sick-Sicker example and its assumed inputs.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1177/0272989x221103163

Cites

44 of 44 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

3 of 9 cited sources checked · 2 supported · 0 flagged · accuracy not yet measured

  • Study design and conductcitation

    The probabilistic sensitivity analysis randomly drew parameter sets from distributions that reflect the current uncertainty in model parameter estimates.

    “To quantify the effect of model parameter uncertainty on cost-effectiveness outcomes, we conducted a PSA by randomly drawing K parameter sets (n_sim) from distributions that reflect the current uncertainty in model parameter estimates.26”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [26] Briggs AH, Weinstein MC, Fenwick EAL, Karnon J, Sculpher MJ, Paltiel AD. Model Parameter Estimation and Uncertainty Analysis: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force Working Group-6. Medical Decision Making. 2012 Sep; 32(5): 722– 32. 22990087 source
  • Findingreasoning

    In the Sick-Sicker model, because the Dead state is absorbing, the proportion of the cohort in that state increases over time.

    “Given that the D state is absorbing, the proportion in this state increases over time.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The cohort is homogeneous within each health state.
    • The authors state this as an assumption: Individuals in the cohort residing in a particular health state have the same characteristics and are indistinguishable from one another.
    • The authors state this as an assumption: In a cSTM, transition probabilities depend only on the current health state in a given cycle and not on the history of past transitions or time spent in a given state (the Markovian assumption).
    • The authors state this as an assumption: The rate of becoming sick is constant over the year, implying that the time until a healthy person becomes sick is exponentially distributed.
    • The authors state this as an assumption: All transitions to non-death states are conditional on surviving to the end of a cycle.
    • The authors state this as an assumption: It is not possible to distinguish between Sick and Sicker patients, so individuals in both disease states receive the treatment under the treatment strategies.
  • Findingown data

    Strategy AB has an ICER of $125,764 per QALY.

    “SoC is the least costly and effective strategy, followed by Strategy B producing an expected incremental benefit of 1.473 QALYs per individual for an additional expected cost of $107,521 with an ICER of $72,988/QALY followed by Strategy AB with an ICER $125,764/QALY.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: All parameters of the example model are known, or at least the characterization of their uncertainty (their distributions) is known.
  • Findingown data

    In the probabilistic sensitivity analysis of the time-independent model, strategy AB has the highest expected costs and QALYs.

    “The CE scatter plot for CEA using the time-independent model shows that strategy AB has the highest expected costs and QALYs.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: All model parameters are independent of each other in the probabilistic sensitivity analysis.
  • Findingown data

    Strategies SoC, B and AB form the cost-effectiveness efficient frontier.

    “Strategies SoC, B and AB form the cost-effectiveness efficient frontier (Figure 3).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: All parameters of the example model are known, or at least the characterization of their uncertainty (their distributions) is known.
  • Classified as probably not a substantial claim. 14
  • May combine two claims. 15
  • The quote only partly states this claim. 1

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    The markovchain package simulates time-independent and time-dependent Markov chains but is not designed to conduct economic evaluations.

    “The markovchain package simulates time-independent, and time-dependent Markov chains but is not designed to conduct economic evaluations.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [10] Spedicato GA. Discrete Time Markov Chains with R. The R Journal [Internet]. 2017; 9(2): 84– 104. Available from: https://journal.r-project.org/archive/2017/RJ-2017-036/RJ-2017-036.pdf source

      The Introduction describes simulating trajectories from a discrete-time Markov chain, and the Non homogeneous Markov chains section describes handling time-dependent chains; its example says rmarkovchain also works on those chains. The Additional information section says the package was designed for common operations on discrete-time Markov chains, but our copy does not explicitly establish that it is not designed to conduct economic evaluations. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Premisecitation

    Model parameters could be correlated or have a rank order, so statistical methods that simulate these correlations or rank order might be needed.

    “However, parameters could be correlated or have a rank order, and appropriate statistical methods that simulate these correlations or rank order might be needed.32”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [32] Goldhaber-Fiebert JD, Jalal HJ. Some Health States Are Better Than Others: Using Health State Rank Order to Improve Probabilistic Analyses. Medical Decision Making [Internet]. 2015; 36(8): 927– 40. Available from: http://mdm.sagepub.com/cgi/doi/10.1177/0272989X15605091 26377369 PMC4794424 source

  • May combine two claims.

    Findingown data

    At willingness-to-pay thresholds greater than or equal to $120,000 per QALY gained, strategy AB has the highest probability of being cost-effective and the highest expected net monetary benefit.

    “This switches to strategy B for WTP thresholds between $80,000 and $120,000 per QALY gained and to strategy AB for WTP thresholds greater than or equal to $120,000 per QALY gained.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: All model parameters are independent of each other in the probabilistic sensitivity analysis.

  • May combine two claims.

    Findingown data

    At willingness-to-pay thresholds below $80,000 per QALY gained, strategy SoC has both the highest probability of being cost-effective and the highest expected net monetary benefit.

    “At WTP thresholds less than $80,000 per QALY gained, strategy SoC has both the highest probability of being cost-effective and the highest expected NMB.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: All model parameters are independent of each other in the probabilistic sensitivity analysis.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    In the Sick-Sicker model, the whole cohort starts in the Healthy state and transitions to the rest of the states over time.

    “The whole cohort starts in the H state and transitions to the rest of the states over time.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The cohort is homogeneous within each health state.
    • The authors state this as an assumption: Individuals in the cohort residing in a particular health state have the same characteristics and are indistinguishable from one another.
    • The authors state this as an assumption: In a cSTM, transition probabilities depend only on the current health state in a given cycle and not on the history of past transitions or time spent in a given state (the Markovian assumption).
    • The authors state this as an assumption: The rate of becoming sick is constant over the year, implying that the time until a healthy person becomes sick is exponentially distributed.
    • The authors state this as an assumption: All transitions to non-death states are conditional on surviving to the end of a cycle.
    • The authors state this as an assumption: It is not possible to distinguish between Sick and Sicker patients, so individuals in both disease states receive the treatment under the treatment strategies.

  • May combine two claims.

    Findingown data

    Strategy B produces an expected incremental benefit of 1.473 QALYs per individual for an additional expected cost of $107,521, giving an ICER of $72,988 per QALY.

    “SoC is the least costly and effective strategy, followed by Strategy B producing an expected incremental benefit of 1.473 QALYs per individual for an additional expected cost of $107,521 with an ICER of $72,988/QALY followed by Strategy AB with an ICER $125,764/QALY.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: All parameters of the example model are known, or at least the characterization of their uncertainty (their distributions) is known.

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    Among the four strategies in the base-case analysis, standard of care (SoC) is the least costly and least effective strategy.

    “SoC is the least costly and effective strategy, followed by Strategy B producing an expected incremental benefit of 1.473 QALYs per individual for an additional expected cost of $107,521 with an ICER of $72,988/QALY followed by Strategy AB with an ICER $125,764/QALY.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: All parameters of the example model are known, or at least the characterization of their uncertainty (their distributions) is known.

  • May combine two claims.

    Findingown data

    At willingness-to-pay thresholds between $80,000 and $120,000 per QALY gained, strategy B has the highest probability of being cost-effective and the highest expected net monetary benefit.

    “This switches to strategy B for WTP thresholds between $80,000 and $120,000 per QALY gained and to strategy AB for WTP thresholds greater than or equal to $120,000 per QALY gained.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: All model parameters are independent of each other in the probabilistic sensitivity analysis.

  • May combine two claims.

    Findingown data

    In the probabilistic sensitivity analysis, standard of care has the lowest expected cost and QALYs.

    “Standard of care has the lowest expected cost and QALYs.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: All model parameters are independent of each other in the probabilistic sensitivity analysis.

  • May combine two claims.

    Findingown data

    In the probabilistic sensitivity analysis, strategy B is more effective and less costly than strategy A.

    “Strategy B is more effective and least costly than Strategy A.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: All model parameters are independent of each other in the probabilistic sensitivity analysis.

  • May combine two claims.

    Premisereasoning

    Shortening the cycle length reduces the biases from discretization but requires simulating the model for a larger number of cycles, which can be computationally burdensome.

    “One approach to reducing these biases is to shorten the cycle length, requiring simulating the model for a larger number of cycles, which can be computationally burdensome.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    cSTMs can also be implemented in R via discrete-time difference equations or continuous-time differential equations, in addition to a discrete-time transition matrix.

    “However, cSTM can also be implemented via (discrete-time) difference equations or (continuous-time) differential equations in R.35,36”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [35] Grimmett G, Welsh D. Markov Chains. In: Probability: An introduction [Internet]. 2nd ed. Oxford University Press; 2014. p. 203– 3. Available from: www.statslab.cam.ac.uk/{~}grg/teaching/chapter12.pdf

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [36] Axler S, Gehring FW, Ribet KA. Difference Equations. In New York, NY: Springer; 2005. Available from: http://link.springer.com/10.1007/0-387-27645-9 source

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Reasonably complex cSTMs can be built in R as long as the computer's RAM can store the size of the transition probability matrix and the outputs of interest.

    “It is possible to build reasonably complex cSTMs in R as long as the computer’s RAM can store the size of the transition probability matrix and outputs of interest.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    All transition probabilities in a transition probability matrix should have values between 0 and 1.

    “The elements of P are the transition probabilities of moving from state i to state j, p[i,j], where {i, j} = 1, …, nS and all should have values between 0 and 1.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    For time-independent cSTMs, the capacity of current RAM in personal computers should generally not be a problem.

    “For time-independent cSTMs, in general, this should not be a problem with the capacity of current RAM in personal computers.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The cohort trace matrix M stores the output of the cSTM and can be used to compute epidemiological and economic outcomes such as life expectancy, prevalence, cumulative resource use, and costs.

    “Thus, M stores the output of the cSTM, which could be used to compute various epidemiological, and economic outcomes, such as life expectancy, prevalence, cumulative resource use, and costs, etc.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    The transition matrices created for the Sick-Sicker model were verified as valid by checking that each row sums to one and that each entry is between 0 and 1.

    “Once all transition matrices are created, we verify they are valid by checking that each row sums to one and that each entry is between 0 and 1.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The general conceptualization of time-independent cSTMs presented in the tutorial should apply to other programming languages such as MATLAB, Python, C++, and Julia.

    “Our general conceptualization should apply to other programming languages (e.g., MATLAB, Python, C++, and Julia).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Cohort state-transition models (cSTMs) are usually computationally less demanding than individual-based state-transition models (iSTMs), providing the ability to conduct probabilistic sensitivity analysis (PSA) and value-of-information (VOI) analyses that otherwise might not be computationally feasible with iSTMs.

    “cSTMs are usually computationally less demanding than individual-based state-transition models (iSTMs), providing the ability to conduct PSA and value-of-information (VOI) analyses that otherwise might not be computationally feasible with iSTMs.2”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [2] Siebert U, Alagoz O, Bayoumi AM, Jahn B, Owens DK, Cohen DJ, State-Transition Modeling: A Report of the ISPOR-SMDM Modeling Good Research Practices Task Force-3. Medical Decision Making [Internet]. 2012; 32(5): 690– 700. Available from: http://mdm.sagepub.com/cgi/doi/10.1177/0272989X12455463 22990084 source

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Hazard ratios apply to rates and not to probabilities.

    “These calculations are required because hazard ratios only apply to rates and not to probabilities.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Setting the seed of the pseudo-random number generator used in sampling parameter values ensures reproducibility of the PSA results.

    “The function input seed sets the seed of the pseudo-random number generator used in sampling parameter values, which ensures reproducibility of the PSA results.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Study design and conductnone given

    For a transition probability matrix to be correctly specified, each row of the matrix must sum to one.

    “For P to be a correctly specified transition probability matrix, each row of the transition probability matrix must sum to one, ∑j=1nSp[i,j]=1 for all i = 1, …, nS.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Although some implementation details presented in the tutorial are specific to the R programming language, much of the code structure shown would be similar in other programming languages.

    “While some of the presented implementation details are specific to the R programming language, much of the code structure shown in this tutorial would be similar in other programming languages.”
    Quote in the paperNot a substantial claim
23 of 23

  • The authors state this as an assumption.

    The cohort is homogeneous within each health state.

    “The cohort is assumed to be homogeneous within each health state.”

  • The authors state this as an assumption.

    Individuals in the cohort residing in a particular health state have the same characteristics and are indistinguishable from one another.

    “Individuals in the cohort residing in a particular health state are assumed to have the same characteristics and are indistinguishable from one another.”

  • The authors state this as an assumption.

    In a cSTM, transition probabilities depend only on the current health state in a given cycle and not on the history of past transitions or time spent in a given state (the Markovian assumption).

    “In a cSTM, the transition probabilities only depend on the current health state in a given cycle, meaning that the transition probabilities do not depend on the history of past transitions or time spent in a given state. This property is often referred to as the “Markovian assumption.””

  • The authors state this as an assumption.

    The rate of becoming sick is constant over the year, implying that the time until a healthy person becomes sick is exponentially distributed.

    “Equation (1) assumes that the rate of becoming sick is constant over the year, implying that the time until a healthy person becomes sick is exponentially distributed.”

  • The authors state this as an assumption.

    All transitions to non-death states are conditional on surviving to the end of a cycle.

    “We assume that all transitions to non-death states are conditional on surviving to the end of a cycle.”

  • The authors state this as an assumption.

    It is not possible to distinguish between Sick and Sicker patients, so individuals in both disease states receive the treatment under the treatment strategies.

    “We assume that it is not possible to distinguish between Sick and Sicker patients; therefore, individuals in both disease states receive the treatment under the treatment strategies.”

  • The authors state this as an assumption.

    All parameters of the example model are known, or at least the characterization of their uncertainty (their distributions) is known.

    “The parameterization of our example model assumes all parameters are known, or at least, the characterization of their uncertainty is known (i.e., we know their distributions).”

  • The authors state this as an assumption.

    All model parameters are independent of each other in the probabilistic sensitivity analysis.

    “Similarly, our PSA analysis simplifies reality where all model parameters are assumed to be independent of each other.”
8 of 8

Cites

Flagged first, then by citations

1 of the 45 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. Rosmalen J van, 2013, Med Decis Making, 33

Cited by

Sorted by citations

Related

Authors
Alarid-Escudero, F., Krijkamp, E., Enns, E. A., Yang, A., Hunink, M. G. M., Pechlivanoglou, P., & Jalal, H.
Affiliations
  • Division of Public Administration, Center for Research and Teaching in Economics (CIDE), Aguascalientes, Aguascalientes, Mexico
  • Erasmus MC (NL)
  • University of Minnesota (US)
  • Hospital for Sick Children (CA)
  • Harvard University (US)
  • University of Toronto (CA)
  • University of Ottawa (CA)
ORCID
Venue
Medical Decision Making
Publisher
SAGE Publications
Published
2022-06-30
Volume
43
Issue
1
Pages
3-20
ISSN
  • 0272-989X (ISSN-L)
  • 1552-681X
DOI
10.1177/0272989x221103163
PMC
PMC9742144
OpenAlex
W3195294569
Cited by
39
References
45
Citations by year
2026: 13 · 2025: 12 · 2024: 8 · 2023: 3 · 2022: 2 · 2021: 1
FWCI
4.93
Type
journal article
Language
en
Open access
Licence
Locations
  • Medical Decision Making (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • EUR Research Repository (Erasmus University Rotterdam) (publishedVersion, other-oa)
  • arXiv (Cornell University) (submittedVersion)
  • PubMed Central (submittedVersion)
  • EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • EUR Research Repository (Erasmus University Rotterdam) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Funders
  • National Institutes of Health (5u01ca253913-05)
  • National Institute on Drug Abuse (K01DA048985)
  • National Cancer Institute (U01-CA253913, U01CA199335, U01CA265750)
  • National Institute of Allergy and Infectious Diseases (K25AI118476)
Topics
  • Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Health Systems, Economic Evaluations, Quality of Life
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Simulation Techniques and Applications
Keywords
decision models, cost-effectiveness analysis, health state transitions, probabilistic sensitivity analysis, R implementation
MeSH terms
Cost-Effectiveness Analysis, Cost-Benefit Analysis, Humans, Markov Chains, Probability, Programming Languages, Software, Quality-Adjusted Life Years

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-16.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-16.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review