CoChat
Sign Up
Cited by 0Medical Image Segmentation TechniquesAdvanced Radiotherapy TechniquesArtificial Intelligence in Healthcare and Education

Uncertainty quantification of U‐Net based segmentation tool using conformal prediction

Open access
Borden, Bailey J., Graham‐Knight, John B., Lasserre, Patricia, Lucas, Sarah, Rajapakshe, Rasika D.·2026·Journal of Applied Clinical Medical Physics
DOI 10.1002/acm2.70663PMC PMC13291182
Open this paper in CoChat — free account

Abstract

Background: The radiation therapy treatment process is very labour intensive, and artificial intelligence (AI) based auto contouring tools are increasingly being adopted to improve efficiency. However, current acceptance testing of AI auto contouring algorithms relies primarily on area‐ and distance‐based metrics, with limited assessment of model uncertainty. Purpose: To demonstrate conformal prediction as a complementary error analysis technique for AI auto contouring algorithms, providing spatially localized uncertainty information that traditional metrics do not capture. Methods: A U‐Net architecture with a ResNet‐34 encoder was trained on BC Cancer breast data to segment the left lung, right lung, and the heart. Initial testing was performed on a subset of 376 computed tomography (CT) scans using both area‐based (IoU) and distance‐based (HD95) metrics. Conformal prediction using adaptive prediction sets was then performed on 138 CT scans. The change in the derivative of the intersection over union (IoU) between the original predictions and the conformal predictions was observed with respect to the selected confidence level. Results: U‐Net achieved a mean IoU of 0.924 and a mean HD95 of 11.35. When conformal prediction was applied using a 90% confidence threshold, the percent differences between the conformal prediction IoUs and the U‐Net prediction IoUs were 1.01%, 0.89%, and 1.46% for the left lung, right lung, and the heart, respectively. The IoU derivatives differed significantly between true positive and false positive structure predictions ( P < 0.001), indicating that conformal prediction can distinguish reliable predictions from unreliable ones. Conclusions: Conformal prediction provides an additional tool for acceptance testing of AI auto contouring algorithms. Beyond traditional area‐ and distance‐based metrics, it spatially localizes uncertain predictions and offers a mechanism for identifying false positives.

AI snapshot

Written from the paper’s full text.

TL;DR

This paper demonstrates conformal prediction with adaptive prediction sets as a post-hoc uncertainty quantification tool for a U-Net auto-contouring model, complementing IoU and HD95 metrics. It shows that conformal uncertainty maps localize low-confidence regions and that derivative analysis can distinguish true positive from false positive structure predictions.

Key findings

  • U-Net achieved a mean IoU of 0.924 and mean HD95 of 11.35 on 376 CT scans; heart had the lowest IoU (0.877) and lowest HD95 (8.77), while left and right lung IoUs were 0.911 and 0.914.
  • At 90% conformal coverage, IoU decreased from 0.874 to 0.861 for heart (1.46% difference), 0.894 to 0.885 for left lung (1.01%), and 0.896 to 0.893 for right lung (0.894%).
  • The derivative of IoU with respect to α differed significantly between true positive and false positive structure predictions (Mann–Whitney U test, P < 0.001), with false positives showing larger relative changes.
  • Thresholding at the 90th percentile of true positive derivative distributions reduced false positives across structures: heart net reduction 42 (136 fewer false positives, 94 more false negatives), left lung net 194 (324 fewer, 130 more), right lung net 120 (249 fewer, 129 more).
  • Of 11,860 slices containing both lung contours, 3,409 (28.7%) had false positive heart predictions.

Methods

  • A U-Net with ResNet-34 encoder, pretrained on ImageNet, was trained on BC Cancer breast radiotherapy CT data to segment left lung, right lung, and heart using 512 × 512 single slices; 62,234 slices for training, 2,063 for validation, and 13,531 for testing, with no data augmentation.
  • Initial testing used IoU and HD95 on a subset of 376 CT scans. Conformal prediction with adaptive prediction sets was performed on 138 CT scans; 26,497 slices were randomly selected, 2,407 slices used for calibration, leaving a final test set of 24,090 slices.
  • The nonconformity measure was the cumulative softmax score of the true class. Primary analysis used α = 0.1 (90% coverage), with α varied over 200 equally spaced values in [0.1, 0.005]. The normalized derivative dIoU/dα was analyzed, and maximum derivatives were compared between true positive and false positive predictions using the Mann–Whitney U test.

Limitations

  • The paper states: a relatively simple U-Net with ResNet-34 was used, and more advanced models like nnU-Net may show different uncertainty characteristics; the methodology has not been validated in clinical practice; reliance on softmax outputs prevents direct application to commercial systems, though morphological prediction sets are suggested as a path forward; and the analysis covers one anatomical site with three structures.
  • The paper also states: the dataset lacked patient demographic information, preventing subgroup analysis; data spanned 2001–2019 with changing contouring practices, likely producing heterogeneous reference contours; thresholds need validation on independent data and prospective validation is needed; and possible out-of-distribution pixels were not analyzed.
  • Evident from the text: the false positive heart rate of 28.7% indicates the model would be unsuitable for clinical adoption without further investigation, and the calibration/test sets share distributional characteristics, so the conformal calibration may absorb some but not all historical contour variability.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1002/acm2.70663

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

14 of 21 cited sources checked · 3 supported · 1 flagged · accuracy not yet measured

  • Interpretationcitation

    The model lacks spatial reasoning to tell air cavities apart from lung tissue.

    “the model lacks spatial reasoning to tell them apart. 29”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [29] Osadebey M, Andersen HK, Waaler D, Fossaa K, Martinsen ACT, and Pedersen M. Three‐stage segmentation of lung region from CT images using deep neural networks. BMC Med Imag. 2021; 21(1): 112. PMC8280386 34266391 source

      The Introduction (page unavailable) says that bowel gas and subcutaneous cavities can have grayscale values similar to the lung region, making automated segmentation challenging. It does not say the model lacks spatial reasoning or is unable to distinguish those structures from lung tissue; the Discussion’s “Reduction of false positives” section instead reports that its CNN classifiers reduce false positives. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
  • Premisecitation

    The training data spans 2001-2019, a period over which contouring practices changed substantially.

    “The training data spans 2001–2019, a period over which contouring practices changed substantially. 35”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [35] Lin D, Lapen K, Sherer MV, et al. A systematic review of contouring guidelines in radiation oncology: analysis of frequency, methodology, and delivery of consensus recommendations. Int J Radiat Oncol Biol Phys. 2020; 107(4): 827‐ 835. 32311418 PMC8262136 source

      The Results (‘Trends over time’; page unavailable) document a substantial increase in contouring guidelines through 2019, and the Conclusions link that increase to growing radiation-treatment complexity. This supports a changing contouring context, but the review does not establish that contouring practices themselves changed substantially over 2001–2019. It also does not describe the citing authors’ training data. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
  • Premisecitation

    Current acceptance testing of AI auto contouring algorithms relies on area- and distance-based metrics that measure overall contour agreement but say nothing about where the model is uncertain.

    “Acceptance testing of AI auto contouring algorithms typically involves both area‐based metrics (e.g., Dice Similarity Coefficient, Intersection over Union (IoU)) and distance‐based metrics (e.g., Hausdorff Distance) evaluated against reference standard contours. 11 , 12 , 13 , 14 These metrics are well established for assessing contour agreement, but they say nothing about where the model is confi”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites [11] Hurkmans C, Bibault JE, Brock KK, et al. A joint ESTRO and AAPM guideline for development, clinical validation and reporting of artificial intelligence models in radiation therapy. Radioth Oncol. 2024; 197: 110345. 38838989 source

      In the Results section on volume segmentation (page unavailable), the guideline recommends reporting Dice Similarity Coefficient and Hausdorff Distance against a ground-truth contour, and says simple geometric validation is insufficient to predict clinical usefulness. It does not state that these metrics reveal nothing about where an auto-contouring model is uncertain; its discussion of model uncertainty concerns dose prediction, not segmentation. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [12] Reinke A, Tizabi M, Sudre C, et al. Common limitations of image processing metrics: a picture story. arXiv preprint arXiv:2104.05642. 2021. source

      The Fundamentals (pp. 23–24) identifies Dice and IoU as overlap-based segmentation metrics and Hausdorff distance as a distance-based metric comparing predicted and reference boundaries. The segmentation discussion (p. 99) distinguishes predicted pixel probabilities from the binarized masks used to calculate metric values, supporting the distinction between contour agreement and model confidence. Our copy does not establish how AI autocontouring acceptance testing is typically conducted or explicitly discuss locating model uncertainty. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [13] Reinke A, Tizabi MD, Baumgartner M, et al. Understanding metric‐related pitfalls in image analysis validation. Nat Methods. 2024; 21(2): 182‐ 194. 38347140 PMC11181963 source

      The Results section ([P2.4], page unavailable) says reference-based metrics compare algorithm output with a reference annotation. The Extended Data identifies Dice and IoU as overlap-based metrics, while Results ([P2.3], page unavailable) identifies Hausdorff distance as distance-based. These passages do not establish what acceptance testing of AI autocontouring typically uses or explicitly address whether these metrics show where a model is uncertain. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
  • Premisecitation

    MC dropout requires 10-50 forward passes at test time.

    “MC dropout requires 10–50 forward passes at test time, 17 which is substantial when processing hundreds of CT slices per patient.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [17] Gal Y, Ghahramani Z. Dropout as a bayesian approximation: representing model uncertainty in deep learning. In Proceedings of the 33rd International Conference on Machine Learning (ICML). PMLR 2016; 48: 1050‐ 1059. source

      Section 4 (p. 4) defines MC dropout as averaging T stochastic forward passes, supporting the test-time procedure. Our copy does not specify that 10–50 passes are required; it also says the passes can run concurrently. (LLM, conf=0.87)

      Experimental — accuracy not yet measured
  • Findingown data

    Uncertain regions cluster along structure boundaries.

    “Uncertain regions appear along the boundaries of all three structures”
    • Also stated in: § DISCUSSION · plain
      “uncertain regions cluster along structure boundaries”
    Quote in the paperSubstantial claim
  • May combine two claims. 19
  • Classified as probably not a substantial claim. 13
  • The quote only partly states this claim. 2

  • May combine two claims.

    Premisecitation

    MC dropout and deep ensembles require either architectural modifications or training and storing multiple models, and neither provides formal statistical guarantees on coverage.

    “Both have been applied to medical image segmentation, 19 but they require either architectural modifications or training and storing multiple models, and neither provides formal statistical guarantees on coverage.”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites [17] Gal Y, Ghahramani Z. Dropout as a bayesian approximation: representing model uncertainty in deep learning. In Proceedings of the 33rd International Conference on Machine Learning (ICML). PMLR 2016; 48: 1050‐ 1059. source

      Section 4 (p. 4) explicitly says MC dropout does not change the dropout neural network model and can use existing models trained with dropout, contrary to the claimed need for architectural modifications. The passages found do not address deep ensembles or formal statistical guarantees on coverage. (LLM, conf=0.84)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • cites [18] Lakshminarayanan B, Pritzel A, Blundell C. Simple and scalable predictive uncertainty estimation using deep ensembles. in Advances in Neural Information Processing Systems (NeurIPS). 2017; 30. source

      The paper reports adding dropout to the network for MC-dropout (Section 3.4, p. 6) and training ensemble networks independently (Algorithm 1, p. 5). It also says an ensemble has M times the parameters of a single network (Discussion, p. 9). It evaluates calibration and observed prediction-interval coverage, but does not state whether either method provides formal statistical coverage guarantees. (LLM, conf=0.87)

      Experimental — accuracy not yet measured
    • cites [19] Jungo A, and Reyes M. Assessing reliability and challenges of uncertainty estimations for medical image segmentation. In Medical Image Computing and Computer Assisted Intervention – MICCAI. 2019; 11765: 48‐ 56, Springer. source

  • May combine two claims.

    Premisecitation

    Conformal prediction is a distribution-free, post-hoc framework that operates on saved model predictions, so it can be applied to any auto contouring algorithm without touching the model.

    “It is a distribution‐free, post‐hoc framework that operates on saved model predictions, so it can be applied to any auto contouring algorithm without touching the model. 21”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [20] Shafer, G, and Vovk, V. A tutorial on conformal prediction. J Mach Learn Res. 2008; 9(3): 1532‐ 4435. source

      The Abstract says conformal prediction can be applied to any method that produces a prediction, and the Introduction says its validity does not depend on the probability distribution under exchangeability. But the tutorial describes an on-line procedure using past examples; it does not establish that conformal prediction operates on saved model predictions, requires no changes to a model, or has been applied to auto-contouring algorithms. (LLM, conf=0.87)

      Experimental — accuracy not yet measured
    • cites [21] Angelopoulos, A. N, and Bates, S. A gentle introduction to conformal prediction and distribution‐free uncertainty quantification. arXiv preprint arXiv: 2107.07511. 2022. source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Deep ensembles train multiple independent models and use disagreement across predictions as an uncertainty estimate.

    “Deep ensembles train multiple independent models and use disagreement across predictions as an uncertainty estimate. 18”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [18] Lakshminarayanan B, Pritzel A, Blundell C. Simple and scalable predictive uncertainty estimation using deep ensembles. in Advances in Neural Information Processing Systems (NeurIPS). 2017; 30. source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The IoU values in Table 1 are consistent with published benchmarks for U-Net-based thoracic OAR segmentation.

    “The IoU values in Table 1 are consistent with published benchmarks for U‐Net‐based thoracic OAR segmentation. 26”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [26] Dong X, Lei Y, Wang T, et al. Automatic multiorgan segmentation in thorax CT images using U‐net‐GAN. Med Phys. 2019; 46(5): 2157‐ 2168. 30810231 PMC6510589 source

      The Results (page not supplied) provide published performance figures for U-Net-GAN segmentation of thoracic organs at risk, and the Discussion compares its performance with other methods. However, the paper reports Dice scores and other metrics, not IoU, so it does not directly establish that the citing paper’s Table 1 IoU values are consistent with its results. (LLM, conf=0.84)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Interpretationcitation

    The heart is the hardest of the three structures to segment in breast radiotherapy data because it is not always fully visible in the CT scan and contouring practices vary across treatment lateralities.

    “This makes sense: it is the hardest of the three structures to segment in breast radiotherapy data because it is not always fully visible in the CT scan and contouring practices vary across treatment lateralities. 27”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [27] Duane F, Aznar MC, Bartlett F, et al. A cardiac contouring atlas for radiotherapy. Radiother Oncol. 2017; 122(3): 416– 422. 28233564 PMC5356506 source

      The Background and Discussion (pages unavailable in our copy) describe cardiac substructures as challenging to contour reproducibly on radiotherapy CT scans. The paper does not compare the heart with the other two structures in the claim, say that the whole heart is not always fully visible, or attribute contouring difficulty to practices varying by treatment laterality. (LLM, conf=0.85)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Premisecitation

    Deep ensembles need multiple independently trained models.

    “Deep ensembles need multiple independently trained models. 18”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [18] Lakshminarayanan B, Pritzel A, Blundell C. Simple and scalable predictive uncertainty estimation using deep ensembles. in Advances in Neural Information Processing Systems (NeurIPS). 2017; 30. source

  • May combine two claims.

    Findingown data

    Thresholding at the 90th percentile of true positive derivative distributions reduced false positives across all three structures, with net error reductions of 42 for the heart, 194 for the left lung, and 120 for the right lung.

    “For the heart, this reduces 136 false positives while introducing 94 false negatives, a net reduction of 42 errors. The lungs show a better trade‐off, with net reductions of 194 (left) and 120 (right).”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    U-Net achieved a mean IoU of 0.924 and a mean HD95 of 11.35 on the test set of 376 CT scans.

    “U‐Net achieved a mean IoU of 0.924 and a mean HD95 of 11.35.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Heart masks had the lowest HD95 values among the three structures.

    “The heart masks have the lowest HD95 values (Table 1).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    At a 90% confidence threshold, the percent differences between conformal prediction IoUs and U-Net prediction IoUs were 1.46% for the heart, 1.01% for the left lung, and 0.894% for the right lung.

    “When conformal prediction was applied using a 90% confidence threshold, the percent differences between the conformal prediction IoUs and the U‐Net prediction IoUs were 1.01%, 0.89%, and 1.46% for the left lung, right lung, and the heart, respectively.”
    • Also stated in: § RESULTS · plain
      “The heart had the largest reduction, with a 1.46 % difference between the full prediction IoU and the conformal prediction IoU.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    U-Net achieved the lowest IoU in the heart and the highest IoU in the lungs.

    “U‐Net achieved the lowest IoU in the heart and the highest IoU in the lungs.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    At α = 0.05, a large portion of the false heart structure was marked as uncertain, and at α = 0.01 it grew further.

    “At α = 0.05, a large portion of this false structure is already marked as uncertain, and at α = 0.01 it grows further.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Conformal prediction spatially localizes uncertain predictions and offers a mechanism for identifying false positives.

    “it spatially localizes uncertain predictions and offers a mechanism for identifying false positives.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Monte Carlo dropout approximates Bayesian inference by applying dropout at test time across multiple forward passes and measuring prediction variance.

    “Monte Carlo (MC) dropout approximates Bayesian inference by applying dropout at test time across multiple forward passes and measuring prediction variance. 17”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [17] Gal Y, Ghahramani Z. Dropout as a bayesian approximation: representing model uncertainty in deep learning. In Proceedings of the 33rd International Conference on Machine Learning (ICML). PMLR 2016; 48: 1050‐ 1059. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The model cannot distinguish sources of variation in heart contouring; it reproduces whatever contours appear in the training data.

    “The model cannot distinguish these sources of variation; it reproduces whatever contours appear in the training data.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Conformal prediction produces pixel-level prediction sets with spatially resolved uncertainty and can be rerun at multiple confidence levels without rerunning the model because it uses saved predictions.

    “For segmentation, it produces pixel‐level prediction sets with spatially resolved uncertainty, and because it uses saved predictions it can be rerun at multiple confidence levels without rerunning the model. 21 , 22”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [21] Angelopoulos, A. N, and Bates, S. A gentle introduction to conformal prediction and distribution‐free uncertainty quantification. arXiv preprint arXiv: 2107.07511. 2022. source
    • cites [22] Cordier, T, Blot, V, Lacombe, L. et al. Flexible and systematic uncertainty estimation with conformal prediction via the MAPIE library. In Conformal and Probabilistic Prediction with Applications, 2023; 204: 549‐ 581 (PMLR, 2023). Version 0.8.6, released April 30, 2019,[Internet]. [cited 2025 March]. Available from: https://github.com/scikit‐learn‐contrib/MAPIE

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The model likely learned a mixture of historical practices rather than any single current guideline, which probably contributes to the false positive heart predictions observed.

    “The model likely learned a mixture of historical practices rather than any single current guideline, which probably contributes to the false positive heart predictions we observed.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Adaptive prediction sets include classes in order of decreasing predicted probability until the cumulative probability exceeds a calibrated threshold.

    “Adaptive prediction sets, a variant for classification tasks, include classes in order of decreasing predicted probability until the cumulative probability exceeds a calibrated threshold. 23”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [23] Romano, Y., Sesia, M, and Candes, E. Classification with valid and adaptive coverage. In Larochelle, H., Ranzato, M. Hadsell, R., Balcan, M., and Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, 3581‐ 3591 (Curran Associates, Inc., 2020).

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The methodology has not been validated in clinical practice.

    “The methodology has not been validated in clinical practice.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Conformal prediction uses a held-out calibration set and its coverage guarantee holds regardless of calibration.

    “Conformal prediction uses a held‐out calibration set instead, and its coverage guarantee holds regardless of calibration. 21”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [21] Angelopoulos, A. N, and Bates, S. A gentle introduction to conformal prediction and distribution‐free uncertainty quantification. arXiv preprint arXiv: 2107.07511. 2022. source

  • May combine two claims. The quote only partly states this claim.

    Interpretationown data

    Conformal prediction with adaptive prediction sets fills the gap in acceptance testing by localizing low-confidence regions and distinguishing true from false positive predictions.

    “We showed that conformal prediction with adaptive prediction sets fills this gap.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationown data

    The limited structure set was dictated by available data.

    “The limited structure set was dictated by available data.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisereasoning

    MC dropout and deep ensembles derive uncertainty from the model's own representations, so if the model is poorly calibrated, the estimates will be too.

    “Both derive uncertainty from the model's own representations, so if the model is poorly calibrated, the estimates will be too.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    CT slices within each data subset all originated from the same CTs, meaning that CT slices from the same patient only existed within a single subset.

    “The CT slices within each data subset all originated from the same CTs, meaning that CT slices from the same patient only existed within a single subset.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationnone given

    Presenting AI predictions without uncertainty information encourages passive acceptance.

    “Presenting AI predictions without uncertainty information encourages passive acceptance.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Conformal prediction maps give clinicians something concrete to scrutinize.

    “Conformal prediction maps give clinicians something concrete to scrutinize.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    The data was collected from multiple centers across British Columbia, but without demographics the study cannot assess whether model performance or uncertainty patterns differ across patient populations.

    “The data was collected from multiple centers across the province of British Columbia, but without demographics we cannot assess whether model performance or uncertainty patterns differ across patient populations.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductreasoning

    All data used in the further exploration of the U-Net model came from the set reserved for testing U-Net and had no direct influence on the model's performance.

    “All data used in the further exploration of the U‐Net model came from the set reserved for testing U‐Net and, as such, had no direct influence on the model's performance.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The analysis covers one anatomical site with three structures.

    “And the analysis covers one anatomical site with three structures.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Morphological prediction sets applied to binary segmentation outputs are one option for adapting conformal prediction to commercial systems.

    “Morphological prediction sets applied to binary segmentation outputs are one option, 31”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [31] Mossina, L, and Friedrich, C. Conformal prediction for image segmentation using morphological prediction sets. In Gee, J. C. et al. (eds.) Medical Image Computing and Computer Assisted Intervention – MICCAI 2025, 78‐ 88 (Springer Nature Switzerland, 2026).

      Not checked in the free run.

30 of 30

  • The paper relies on this without support.

    Commercial auto contouring products may not expose softmax scores.

    “Commercial auto contouring products may not expose softmax scores, which means a different nonconformity score would be needed.”

  • The paper relies on this without support.

    The calibration and test sets share the same distributional characteristics.

    “because the calibration and test sets share the same distributional characteristics”

  • The paper relies on this without support.

    Contour deviations near dose gradients affect calculated OAR doses even when overall agreement is high.

    “Contour deviations near dose gradients affect calculated OAR doses even when overall agreement is high.”

  • The paper relies on this without support.

    A contour error in a high-dose gradient region carries more risk than one in a flat-dose region, even if both contribute equally to aggregate IoU.

    “A contour error in a high‐dose gradient region carries more risk than one in a flat‐dose region, even if both contribute equally to aggregate IoU.”
4 of 4

Cites

Flagged first, then by citations

Research works printed but not linked: 4

  1. Mihailidis DN, 2022, Khan's Treatment Planning in Radiation Oncology, 420
  2. Cordier T Blot V Lacombe L. et al.Flexible and systematic uncertainty estimation with conformal predictionviathe MAPIE library. InConformal and Probabilistic Prediction with Applications 2023;204:549‐581(PMLR 2023). Version 0.8.6 released April 30 2019 [Internet]. [cited 2025 March]. Available from:https://github.com/scikit‐learn‐contrib/MAPIE
  3. Romano Y., 2020, Advances in Neural Information Processing Systems, 3581
  4. McDuff S. G., 2022, Khan's Treatment Planning in Radiation Oncology, 134

Authors
Borden, B. J., Graham‐Knight, J. B., Lasserre, P., Lucas, S., & Rajapakshe, R. D.
Affiliations
  • BC Cancer Agency (CA)
  • The University of British Columbia ‐ Okanagan Campus Kelowna British Columbia Canada
  • University of British Columbia, Okanagan Campus (CA)
ORCID
Lasserre, Patricia
Venue
Journal of Applied Clinical Medical Physics
Publisher
Wiley
Published
2026-06-23
Volume
27
Issue
7
Pages
e70663-e70663
ISSN
1526-9914 (ISSN-L)
DOI
10.1002/acm2.70663
PMC
PMC13291182
OpenAlex
W7165780586
Semantic Scholar
ce98f698f638deb847f7f2bf8f1a535d24bd7098
Cited by
0
References
23
FWCI
0.00
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Applied Clinical Medical Physics (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (submittedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Physical Sciences › Physics and Astronomy › Radiation › Advanced Radiotherapy Techniques
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
conformal prediction, uncertainty quantification, U-Net, breast cancer, lung segmentation, cardiac image segmentation, CT images, acceptance testing
MeSH terms
Algorithms, Artificial Intelligence, Breast Neoplasms, Female, Humans, Image Processing, Computer-Assisted, Radiotherapy Dosage, Radiotherapy Planning, Computer-Assisted, Tomography, X-Ray Computed, Uncertainty, Radiotherapy, Intensity-Modulated

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-21.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-21.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review