CoChat
Sign Up
PreprintCited by 1Ethics and Social Impacts of AIExplainable Artificial Intelligence (XAI)Machine Learning and Algorithms

The Coverage-Deferral Trade-Off: Fairness Implications of Conformal Prediction in Human-in-the-Loop Decision Systems

Open access
Tasar, Davut Emre·2025·Preprints.org
DOI 10.20944/preprints202512.2631.v1
Open this paper in CoChat — free accountOpen PDF

Abstract

Conformal prediction (CP) provides distribution-free uncertainty quantification by constructing prediction sets with guaranteed coverage. In human-in-the-loop (HITL) decision systems, these sets naturally define deferral policies: cases with singleton sets proceed automatically, while those with multiple labels require human review. Mondrian CP, which calibrates separately per group, has been proposed to achieve group-conditional coverage validity, ensuring each demographic group meets the target coverage level. However, we demonstrate through extensive experiments (832K evaluations across 14K configurations, 6 datasets, 100 seeds) that improving coverage validity comes at a significant cost: Mondrian CP increases deferral disparity by 143% compared to global CP, despite reducing coverage disparity by 26% on average. This coverage-deferral trade-off is fundamental: it persists across all datasets (p < 0.001), is invariant to HITL parameters, and exhibits monotonic behavior with respect to the shrinkage interpolation parameter γ. We prove an analogous impossibility result for conformal prediction: under specific conditions, coverage parity and deferral parity cannot be simultaneously achieved when base rates differ between groups. We further demonstrate that standard fairness metrics (Equalized Odds, Average Odds Difference) are invariant to CP method choice, identifying deferral gap as a critical operational fairness metric that captures CP’s unique impact on who receives human review, a dimension invisible to standard EO metrics. Our findings provide actionable guidance: use Mondrian for group-conditional coverage validity, global CP for deferral fairness, or shrinkage for balanced trade-offs.

AI snapshot

Written from the paper’s full text.

TL;DR

Mondrian conformal prediction improves group-conditional coverage but worsens deferral fairness in human-in-the-loop systems, revealing a coverage-deferral trade-off. The paper argues this trade-off is structural, invariant to HITL parameters, and implies an impossibility result when base rates differ.

Key findings

  • Mondrian CP reduces coverage gap by 26% on average (0.032 → 0.023) but increases deferral gap by 143% (0.052 → 0.125) compared with global CP, with all differences significant at p < 0.001 across 100 seeds and 6 datasets.
  • The trade-off persists across all datasets, is invariant to HITL parameters (ratio ≈ 2.4×), and is monotonic in the shrinkage parameter γ; the paper reports no optimal γ where both coverage and deferral gaps are minimized.
  • Standard EO metrics (EOD, AOD, FPR/FNR gaps) are invariant to CP method choice because CP changes prediction sets, not point predictions; deferral gap is proposed as the CP-specific fairness metric capturing who receives human review.
  • The paper proves an analogous impossibility result: under specific conditions, coverage parity and deferral parity cannot be simultaneously achieved when base rates differ between groups.

Methods

  • The study runs 832K evaluations across 14K configurations, 6 datasets (Adult, COMPAS, German Credit, Taiwan Credit, ACS Income, Bank Marketing), 5 model types (GB, RF, XGB, LR, MLP), 100 seeds, and α = 0.10, using a 4-way split (60% train, 10% calibration, 10% CP, 20% test).
  • It compares Global, Mondrian, and Shrinkage CP (γ ∈ [0, 1]), with supplementary LAC and SAPS comparisons; split conformal prediction uses an inverse-probability conformity score, and deferral is defined as |Ĉ(x)| > 1.
  • HITL simulation varies human accuracy h ∈ [0.7, 1.0] and review rate r ∈ [0, 1]; statistical robustness uses 100 random seeds, bootstrap 95% CIs, and paired t-tests.

Limitations

  • Stated: binarized sensitive attributes may obscure within-group heterogeneity; intersectional groups (e.g., Sex × Race) show 10–29% higher deferral disparity than single-attribute groups, and individual-level fairness is not addressed.
  • Stated: the HITL simulation assumes group-invariant human accuracy; if h₀ ≠ h₁, Mondrian’s higher deferral rate can amplify accuracy disparity by up to 2.2×. Small calibration sets are noisy (German Credit N = 200; n_g < 30), and the paper recommends n_g ≥ 50 per group.
  • Stated: the method scope is threshold-based APS-style CP; formal coverage guarantees hold only at γ = 0 and γ = 1, not for intermediate γ; Mondrian requires the sensitive attribute at test time, and capacity constraints and multi-class extensions are left for future work.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.20944/preprints202512.2631.v1

Cites

1 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 7 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Mondrian CP increases deferral disparity by 143% compared to global CP.

    “Mondrian CP increases deferral disparity by 143% compared to global CP, despite reducing coverage disparity by 26% on average.”
    • Also stated in: § Coverage-Deferral Trade-off. · plain
      “Mondrian CP reduces coverage disparity by 26% but increases deferral disparity by 143% compared to global CP (p < 0.001, 100 seeds, 6 datasets).”
    • Also stated in: § 7. Conclusion · plain
      “Mondrian CP, designed to target coverage parity via group-conditional validity, increases deferral disparity by 143% compared to global CP.”
    • Also stated in: § 7. Conclusion · plain
      “Increases deferral gap by 143% on average (0.052 → 0.125)”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The main results assume uniform human accuracy (h0 = h1).
  • Findingown data

    German Credit under Mondrian: Group 0 defers 39.1% vs Group 1 at 14.4%.

    “e.g., German Credit under Mondrian: Group 0 defers 39.1% vs Group 1 at 14.4%”
    Quote in the paperSubstantial claim
  • Findingown data

    Final HITL output fairness is approximately invariant to CP method choice.

    “The results confirm that FPR and FNR gaps on y f inal are approximately invariant to CP method choice.”
    Quote in the paperSubstantial claim
  • Findingown data

    Mondrian CP reduces coverage disparity by 26% on average compared to global CP.

    “Mondrian CP increases deferral disparity by 143% compared to global CP, despite reducing coverage disparity by 26% on average.”
    • Also stated in: § Coverage-Deferral Trade-off. · plain
      “Mondrian CP reduces coverage disparity by 26% but increases deferral disparity by 143% compared to global CP (p < 0.001, 100 seeds, 6 datasets).”
    Quote in the paperSubstantial claim
  • Findingown data

    The coverage-deferral trade-off exhibits monotonic behavior with respect to the shrinkage interpolation parameter γ.

    “This coverage-deferral trade-off is fundamental: it persists across all datasets (p < 0.001), is invariant to HITL parameters, and exhibits monotonic behavior with respect to the shrinkage interpolation parameter γ.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 25
  • The quote only partly states this claim. 1
  • Classified as probably not a substantial claim. 11

  • May combine two claims.

    Findingreasoning

    Mondrian reduces coverage gap on average, driven primarily by Adult, but three datasets show increased coverage gaps (COMPAS, German, Taiwan).

    “While Mondrian reduces coverage gap on average (driven primarily by Adult), three datasets show increased coverage gaps (COMPAS: 0.018→0.033,German: 0.035→0.057,Taiwan: 0.011→0.013).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    German Credit has only N = 200 test samples, making Mondrian quantile estimates noisy when group calibration sets are small (n_g < 30).

    “German Credit has only N = 200 test samples, making Mondrian quantile estimates noisy when group calibration sets are small (n g < 30).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    CP-specific fairness metrics are invariant to (h, r); only final accuracy changes.

    “Because CP changes who decides (deferral) rather than underlying point predictions, CP-specific fairness metrics are invariant to (h, r); only final accuracy changes.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    At 10% review capacity, accuracy disparity increases by 40% for Global and 37% for Mondrian compared to unconstrained review.

    “This simulation shows that at 10% review capacity, accuracy disparity increases by 40% for Global and 37% for Mondrian compared to unconstrained review, validating the operational importance of deferral fairness beyond routing alone.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: In the fixed-budget simulation, when deferred cases exceed review capacity, cases are selected uniformly at random (no priority queue) and unreviewed cases receive a default reject decision.
    • Rests on an assumption: Human review is capacity-constrained, and higher deferral rates translate to longer queues.

  • May combine two claims.

    Findingnone given

    The proof sketch finds c ∈ [0.62, 1.84] with mean c ≈ 0.99, confirming comparable classifier uncertainty across groups.

    “finding c ∈ [0.62, 1.84] with mean c ≈ 0.99, confirming comparable classifier uncertainty across groups and supporting the theoretical trade-off.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Mondrian routes 2-3× more cases from one group to human review.

    “A practitioner comparing Global vs Mondrian CP using only TPR gap and FPR gap would conclude”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Mondrian reduces coverage gap by 26% (0.032 → 0.023), though it increases in 3 of 6 datasets.

    “Reduces coverage gap by 26% (0.032 → 0.023), though increases in 3 of 6 datasets”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The main results assume uniform human accuracy (h0 = h1).

  • May combine two claims.

    Findingown data

    Enforcing group-conditional calibration via Mondrian CP can meaningfully reduce coverage disparity, but typically increases deferral disparity.

    “Our experiments show that enforcing group-conditional calibration via Mondrian CP can meaningfully reduce coverage disparity, but typically increases deferral disparity, revealing an inherent design trade-off.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The main results assume uniform human accuracy (h0 = h1).

  • May combine two claims.

    Findingown data

    As γ increases, coverage gap moves toward the Mondrian value while deferral gap consistently increases; there is no sweet spot where both are minimized.

    “The relationship is monotonic on average: as γ increases, coverage gap moves toward the Mondrian value (decreasing in most datasets, increasing in some like COMPAS) while deferral gap consistently increases.There is no”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The main results assume uniform human accuracy (h0 = h1).

  • May combine two claims.

    Findingown data

    Bank Marketing's ratio is inflated by near-zero Global deferral gap; for this dataset, the absolute increase Δ = 0.030 is the appropriate metric.

    “† Bank Marketing's ratio is inflated by near-zero Global deferral gap (0.000 ± 0.001); for this dataset, the absolute increase ∆ = 0.030 is the appropriate metric.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Mondrian CP requires access to the sensitive attribute G at test time for group-conditional threshold selection.

    “Mondrian CP requires access to the sensitive attribute G at test time for group-conditional threshold selection.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Mondrian provides group-conditional validity, not guaranteed equal realized coverage.

    “This occurs because Mondrian provides group-conditional validity (Eq.5), not guaranteed equal realized coverage.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [36]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    Standard EO metrics are computed from the confusion matrix of binary predictions, which is unchanged by CP.

    “Standard EO metrics [17] are computed from the confusion matrix of binary predictions ŷ = ⊮[ f (x) 1 > 0.5], which is unchanged by CP.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [17]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Study design and conductnone given

    The fairness metrics focus on statistical parity and do not address individual-level fairness concerns.

    “Additionally, our fairness metrics focus on statistical parity; individual-level fairness concerns are not addressed.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Cresswell et al. [8] found in human-subject experiments that giving people equalized-coverage prediction sets increased disparate impact compared to standard coverage.

    “recent work by Cresswell et al. [8] sounds an alarm: in human-subject experiments, giving people equalized-coverage prediction sets increased disparate impact compared to standard coverage.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Cresswell et al. [8]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning

    Small differences in final HITL output fairness arise from stochastic HITL simulation because different deferral patterns lead to different cases being reviewed, but aggregate fairness properties remain stable because human accuracy is group-invariant by construction.

    “Small differences (within 0.01) arise from the stochastic HITL simulation: different deferral patterns lead to different cases being reviewed, but the aggregate fairness properties remain stable because human accuracy is group-invariant by construction.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: Human accuracy is group-invariant by construction.

  • May combine two claims.

    Interpretationown data

    Groups with higher deferral rates are exposed to more human decisions, and if human accuracy is lower for that group, final accuracy gaps increase.

    “Groups with higher deferral rates are exposed to more human decisions, and if human accuracy is lower for that group, final accuracy gaps increase.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Shrinkage with γ ∈ [0.25, 0.5] offers practical coverage validity while balancing the trade-off.

    “Thus, shrinkage with γ ∈ [0.25, 0.5] offers practical coverage validity while balancing the trade-off, though formal guarantees apply only at γ = 0 (Global) and γ = 1 (Mondrian).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given

    Real deployments may differ from the experiments: human accuracy varies by group and case difficulty, review capacity is constrained, and institutional factors affect deferral decisions.

    “Real deployments may differ: human accuracy varies by group and case difficulty, review capacity is constrained, and institutional factors affect deferral decisions.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The invariance results (Section 5.4) hold structurally, but magnitudes may vary in practice.

    “The invariance results (Section 5.4) hold structurally, but magnitudes may vary in practice.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given

    Continuous or multi-valued sensitive attributes may exhibit different dynamics.

    “Continuous or multi-valued sensitive attributes may exhibit different dynamics.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    The 100-seed bootstrap provides robust aggregate estimates.

    “While our 100-seed bootstrap provides robust aggregate estimates, practitioners should exercise caution applying Mondrian CP to small datasets.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Practitioners cannot tune away the coverage-deferral trade-off by adjusting review rates; they must choose a CP method (γ value) that aligns with their fairness priorities.

    “Practitioners cannot”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The theoretical trade-off (Theorem 1) extends directly to multi-class settings: Mondrian's group-specific thresholds will still induce differential deferral rates across groups, though the magnitude may vary with |Y|.

    “The theoretical trade-off (Theorem 1) extends directly: Mondrian's group-specific thresholds will still induce differential deferral rates across groups, though the magnitude may vary with |Y |.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Intersectional fairness (multiple attributes) further complicates this, as calibration set sizes shrink rapidly with the number of intersecting groups.

    “We note that intersectional fairness (multiple attributes) further complicates this, as calibration set sizes shrink rapidly with the number of intersecting groups.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisenone given

    Global CP provides marginal coverage but coverage may vary across demographic groups.

    “Standard (”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Other CP scores (e.g., RAPS, confidence-based) may exhibit different trade-off characteristics.

    “Other CP scores (e.g., RAPS, confidence-based) may exhibit different trade-off characteristics.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The empirically recommended range γ ∈ [0.25, 0.5] corresponds approximately to λ ∈ [0.3, 0.7], values where neither objective dominates.

    “Our empirically recommended range γ ∈ [0.25, 0.5] corresponds approximately to λ ∈ [0.3, 0.7], values where neither objective dominates.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    Binarized sensitive attributes may obscure within-group heterogeneity.

    “We use binarized sensitive attributes (e.g., sex, age threshold), which may obscure within-group heterogeneity.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The framework naturally extends to multi-class settings by deferring when |C(x)| > k for practitioner-specified budget k.

    “For multi-class settings (|Y | > 2), our framework naturally extends to |C(x)| > k for practitioner-specified budget k.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Shrinkage provides a heuristic trade-off between Global and Mondrian, but intermediate γ values lack formal coverage guarantees.

    “While shrinkage provides a heuristic trade-off between Global and Mondrian, intermediate γ values lack formal coverage guarantees.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given

    For shrinkage parameter γ in (0,1), formal finite-sample coverage guarantees do not hold, though empirical coverage remains valid.

    “Note: for γ ∈ (0, 1), formal finite-sample coverage guarantees do not hold, though empirical coverage remains valid (Section 3.3).”
    • Also stated in: § Method Scope. · plain
      “For intermediate shrinkage values (γ ∈ (0, 1)), formal coverage guarantees do not hold, though empirical coverage remains valid (Section 3.3).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Mondrian CP calibrates separately per group, achieving group-conditional coverage validity.

    “Mondrian CP [36] addresses this by calibrating separately per group, achieving group-conditional coverage validity, i.e., P[Y ∈ Ĉ(x)|G = g] ≥ 1α for each group g.”
    • Also stated in: § 3.2. Mondrian Conformal Prediction · plain
      “Mondrian CP [36] calibrates separately per group:”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [36]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning

    The analogy to Kleinberg et al. is direct: Mondrian CP targets coverage parity via group-conditional validity, but this necessitates different thresholds per group, leading to different set sizes and hence different deferral rates.

    “The analogy to Kleinberg et al. is direct: Mondrian CP targets coverage parity via groupconditional validity (analogous to calibration, where predictions mean the same thing across groups), but this necessitates different thresholds per group, leading to different set sizes and hence different deferral rates (analogous to unequal error rates).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kleinberg et al.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data

    For aggressive coverage levels (α > 0.2), empty sets become more common and the deferral definition choice matters more.

    “For aggressive coverage levels (α > 0.2), empty sets become more common and the deferral definition choice matters more.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The deferral rule |Ĉ(x)| > 1 captures exactly the cases where the model cannot distinguish between classes at the target confidence level.

    “The deferral rule | Ĉ(x)| > 1 captures exactly the cases where the model cannot distinguish between classes at the target confidence level.”
    Quote in the paperNot a substantial claim
36 of 36

  • The authors state this as an assumption.

    Human accuracy is group-invariant by construction.

    “because human accuracy is group-invariant by construction.”

  • The paper relies on this without support.

    Human decisions are independent of CP method.

    “Since human decisions are independent of CP method, and non-deferred cases use invariant point predictions, final output EO is primarily determined by HITL protocol parameters, not CP choice.”

  • The authors state this as an assumption.

    The main results assume uniform human accuracy (h0 = h1).

    “Our main results assume uniform human accuracy (h 0 = h 1 ).”
    “our HITL simulation assumes group-invariant human accuracy h”

  • The authors state this as an assumption.

    The analysis assumes unbounded human review capacity.

    “Our analysis assumes unbounded human review capacity.”

  • The authors state this as an assumption.

    In the fixed-budget simulation, when deferred cases exceed review capacity, cases are selected uniformly at random (no priority queue) and unreviewed cases receive a default reject decision.

    “when deferred cases exceed review capacity, cases are selected uniformly at random (no priority queue); unreviewed cases receive a default reject decision.”

  • The paper relies on this without support.

    Human review is capacity-constrained, and higher deferral rates translate to longer queues.

    “In practice, human review is capacity-constrained, and higher deferral rates translate to longer queues.”
6 of 6

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Final-output FNR differences exceed the stated 0.01 bound

    Minor

    The text says differences in final-output fairness are within 0.01, but Table 3 reports FNR gaps of 0.055 and 0.073-a difference of 0.018-and gives their maximum difference as 0.02.

1 of 1

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 3

  1. Jiaye Huang, Huazhen Xi, Linjun Zhang, and Rina Foygel Barber. Conformal prediction with learned features. Journal of Machine Learning Research, 25:1-45, 2024. SAPS: Sorted Adaptive Prediction Sets.
  2. Jing Lei and Larry Wasserman. Distribution-free predictive inference for regression. Journal of the American Statistical Association, 109(507):1094-1114, 2014. doi: 10.1080/01621459.2014.881792. Proved exact conditional coverage impossible distribution-free. · DOI
  3. Harris Papadopoulos. Inductive conformal prediction: Theory and application to neural networks. Tools in Artificial Intelligence, pages 315-330, 2008. ICP theory and neural network applications.

Cited by

Sorted by citations

Related

Authors
Tasar, D. E.
Corresponding author
Tasar, Davut Emre
ORCID
Tasar, Davut Emre
Venue
Preprints.org
Publisher
MDPI AG
Published
2025-12-30
DOI
10.20944/preprints202512.2631.v1
OpenAlex
W7117697466
Cited by
1
References
0
Citations by year
2026: 1
Type
posted content
Language
en
Open access
Licence
Locations
Preprints.org (acceptedVersion, cc-by)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
Keywords
conformal prediction, Mondrian conformal prediction, algorithmic fairness, impossibility results, equalized odds, uncertainty quantification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-30.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-30.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review