PreprintCited by 4Medical Image Segmentation TechniquesArtificial Intelligence in Healthcare and EducationAdvanced Neural Network Applications

ConfIC-RCA: Statistically Grounded Efficient Estimation of Segmentation Quality

Open access
Cosarinsky, Matias, Billot, Ramiro, Mansilla, Lucas, Jimenez, Gabriel, Gaggión, Nicolas, Fu, Guanghui, Tirer, Tom, Ferrante, Enzo·2025·arXiv

Abstract

Assessing the quality of automatic image segmentation is crucial in clinical practice, but often very challenging due to the limited availability of ground truth annotations. Reverse Classification Accuracy (RCA) is an approach that estimates the quality of new predictions on unseen samples by training a segmenter on those predictions, and then evaluating it against existing annotated images. In this work we introduce ConfIC-RCA (Conformal In-Context RCA), a novel method for automatically estimating segmentation quality with statistical guarantees in the absence of ground-truth annotations, which consists of two main innovations. First, In-Context RCA, which leverages recent in-context learning models for image segmentation and incorporates retrieval-augmentation techniques to select the most relevant reference images. This approach enables efficient quality estimation with minimal reference data while avoiding the need of training additional models. Second, we introduce Conformal RCA, which extends both the original RCA framework and In-Context RCA to go beyond point estimation. Using tools from split conformal prediction, Conformal RCA produces prediction intervals for segmentation quality providing statistical guarantees that the true score lies within the estimated interval with a user-specified probability. Validated across 10 different medical imaging tasks in various organs and modalities, our methods demonstrate robust performance and computational efficiency, offering a promising solution for automated quality control in clinical workflows, where fast and reliable segmentation assessment is essential. The code is available at https://github.com/mcosarinsky/Conformal-In-Context-RCA

AI snapshot

Written from the paper’s full text.

TL;DR

ConfIC-RCA combines In-Context RCA, using few-shot segmentation models as reverse classifiers with retrieval-augmented reference selection, and Conformal RCA, which uses split conformal prediction to produce statistically grounded intervals for segmentation quality without ground truth. Across 10 medical imaging tasks, it matches or improves traditional atlas-based RCA while being up to two orders of magnitude faster and providing coverage-calibrated quality intervals.

Key findings

  • In-Context RCA with SAM 2 and UniverSeg was much faster than atlas-based RCA: atlas took on average about 1 minute per image, while SAM 2 took 0.70 seconds and UniverSeg 0.37 seconds, approximately 85× and 162× faster, respectively.
  • Retrieval augmentation improved quality estimation with fewer reference samples; on 3D-IRCAdB and JSRT, retrieval-augmented selection achieved better correlation and lower MAE than random selection as reference size increased.
  • On NuCLS histopathology, the SAM 2 variant achieved a correlation of about 0.76 while atlas methods completely failed.
  • Conformal RCA produced prediction intervals with target coverage 90% (α = 0.1); empirical coverage varied by dataset and method, e.g. SAM 2 ranged from 82.1% on PH2 to 94.8% on WBC CV, while PHiSeg (CQR) produced tighter intervals than the prior PHiSeg conformal procedure.
  • In controlled experiments where calibration and test sets were drawn from the same distribution, empirical marginal coverage surpassed the desired 90% target, supporting the role of exchangeability.

Methods

  • Design: ConfIC-RCA has two components: In-Context RCA, which uses in-context/few-shot segmentation models (UniverSeg and SAM 2) as reverse classifiers, and Conformal RCA, which extends RCA with split conformal prediction inspired by Conformalized Quantile Regression but estimates empirical quantiles directly from the RCA score distribution.
  • Data: Evaluated on 10 medical imaging datasets—SCD, HC18, PSFHS, PH2, ISIC 2018, NuCLS, JSRT, 3D-IRCAdB, WBC-CV, and WBC-JTSC—covering diverse organs and modalities; images were converted to grayscale and resized to 256×256, with 3D images sliced to keep slices containing the ROI.
  • Analysis: Segmentation quality was measured mainly with DSC, with additional ASSD and HD95 analyses; experiments compared In-Context RCA, retrieval-augmented RCA, atlas-based RCA, PHiSeg conformal baselines, MC Dropout, and TTA using correlation, MAE, prediction interval width, empirical coverage, bootstrapped confidence intervals, and paired significance tests.

Limitations

  • The paper states that Conformal RCA relies on exchangeability between calibration and test sets; when this is violated by domain shifts in scanner settings or patient populations, coverage can degrade and reliability may suffer in real-world clinical deployment.
  • The paper reports that neither Conformal-RCA nor PHiSeg variants consistently reached the target 90% marginal coverage, likely due to exchangeability violations from domain shifts and heterogeneous data sources; controlled same-distribution experiments achieved nominal coverage.
  • The paper states that validation on real deployment data and human-in-the-loop studies would be needed to establish clinical reliability and practical utility; it also notes that coverage improves as calibration set size increases, with smaller calibration sets showing lower coverage in Table III.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2503.04522

Cites

65 of 65 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

20 of 34 cited sources checked · 14 supported · 0 flagged · accuracy not yet measured

  • Premisecitation
    § I. INTRODUCTION · p. 1

    The distribution of RCA scores over the reference set serves as a proxy for the quality of the original segmentation.

    “The distribution of scores over the reference set serves as a proxy for the quality of the original segmentation.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [5] [5] V. V. Valindria, I. Lavdas, W. Bai, K. Kamnitsas, E. O. Aboagye, A. G. Rockall, D. Rueckert, and B. Glocker, 'Reverse classification accuracy: Predicting segmentation performance in the absence of ground truth,' IEEE transactions on medical imaging , vol. 36, no. 8, p. 1597-1606, 2017. - source

      The Results (p. 5) show distributions of scores across the reference set, but say that selecting the maximum score is the proxy for segmentation quality. Predicting Segmentation Accuracy (p. 4) likewise specifies the maximum, noting that mean and median scores are not useful. The claim captures the use of reference-set scores but is imprecise about which aspect serves as the proxy. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
  • Interpretationcitation
    § I. INTRODUCTION · p. 2

    Conformal RCA leverages split conformal prediction to produce statistically grounded confidence intervals for segmentation quality scores.

    “At the core of our approach is Conformal RCA , which leverages split conformal prediction [6]-[8] to produce statistically grounded confidence intervals for segmentation quality scores.”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites [6] [6] V. Vovk, A. Gammerman, and G. Shafer, Algorithmic Learning in a Random World . Springer-Verlag, 2005. - source
    • cites [7] [7] A. N. Angelopoulos and S. Bates, 'A gentle introduction to conformal prediction and distribution-free uncertainty quantification,' 2022. - source
    • cites [8] [8] Y. Romano, E. Patterson, and E. Candes, 'Conformalized quantile regression,' in Advances in Neural Information Processing Systems , vol. 32. Curran Associates, Inc., 2019. - source
  • Findingown data
    § A. Datasets & Computing Resources · p. 5

    Unlike previous studies that explored RCA focusing on cardiac MRI or multi-organ segmentation on whole-body MRI only, this work broadens the analysis to a wider range of medical imaging challenges.

    “Unlike previous studies that explored RCA focusing on cardiac MRI [28], [29] or multi-organ segmentation on whole-body MRI [5] only, we broaden our analysis to a wider range of medical imaging challenges.”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites [28] [28] R. Robinson et al. , 'Automatic quality control of cardiac mri segmentation in large-scale population imaging,' in MICCAI 2017 . Springer, 2017, pp. 720-727. - source
    • cites [29] [29] --, 'Real-time prediction of segmentation quality,' in MICCAI 2018 . Springer, 2018, pp. 578-585. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [5] [5] V. V. Valindria, I. Lavdas, W. Bai, K. Kamnitsas, E. O. Aboagye, A. G. Rockall, D. Rueckert, and B. Glocker, 'Reverse classification accuracy: Predicting segmentation performance in the absence of ground truth,' IEEE transactions on medical imaging , vol. 36, no. 8, p. 1597-1606, 2017. - source
  • Interpretationcitation
    § B. Proposed Method · p. 5

    Assuming exchangeability between the test sample and calibration set, the constructed interval provides the marginal coverage guarantee stated in Theorem 3.

    “Assuming exchangeability between the test sample and calibration set, the constructed interval provides the marginal coverage guarantee stated in Theorem 3, as shown in [8].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [8] [8] Y. Romano, E. Patterson, and E. Candes, 'Conformalized quantile regression,' in Advances in Neural Information Processing Systems , vol. 32. Curran Associates, Inc., 2019. - source
    • The authors state this as an assumption: Calibration samples and test samples are exchangeable (e.g., i.i.d.).
  • Findingown data
    § C. Evaluation of In-Context RCA · p. 7

    Cosine similarity showed the best results among the considered similarity metrics (Euclidean distance and inner-product).

    “Other similarity metrics, such as Euclidean distance and inner-product were also considered, but cosine similarity showed the best results.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. 16
  • May combine two claims. 44
  • The quote only partly states this claim. 1
  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation
    § I. INTRODUCTION · p. 1

    RCA works by using the predicted segmentation of a new image as pseudo-ground truth to train a reverse classifier, applying it to reference images with known ground truth, and evaluating performance with metrics like DSC.

    “It works by using the predicted segmentation of a new image as a pseudo-ground truth to train a reverse classifier (a segmentation network in our case) 1 . This model is then applied to a set of reference images with known GT, and its performance is evaluated using standard metrics like the DSC.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [5] [5] V. V. Valindria, I. Lavdas, W. Bai, K. Kamnitsas, E. O. Aboagye, A. G. Rockall, D. Rueckert, and B. Glocker, 'Reverse classification accuracy: Predicting segmentation performance in the absence of ground truth,' IEEE transactions on medical imaging , vol. 36, no. 8, p. 1597-1606, 2017. - source
  • Classified as probably not a substantial claim.

    Premisecitation
    § I. INTRODUCTION · p. 1

    Reverse Classification Accuracy (RCA) is a framework designed to predict the performance of an image segmentation method when ground-truth masks are unavailable.

    “Reverse Classification Accuracy (RCA) [5] is a framework originally designed to predict the performance of an image segmentation method in scenarios where GT masks are unavailable.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [5] [5] V. V. Valindria, I. Lavdas, W. Bai, K. Kamnitsas, E. O. Aboagye, A. G. Rockall, D. Rueckert, and B. Glocker, 'Reverse classification accuracy: Predicting segmentation performance in the absence of ground truth,' IEEE transactions on medical imaging , vol. 36, no. 8, p. 1597-1606, 2017. - source
  • May combine two claims.

    Premisecitation
    § B. Experimental Setup · p. 7

    DINOv2 has shown strong generalization, making it effective for general tasks, including those in medical imaging.

    “While DINOv2 was not explicitly trained for the medical domain, it has shown strong generalization, making it effective for general tasks, including those in medical imaging.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [54] [54] M. Oquab et al. , 'Dinov2: Learning robust visual features without supervision,' 2024. - source

      The Introduction and Future work and Discussion report strong performance across varied vision benchmarks and describe DINOv2’s features as transferable across image domains. Our copy does not report a medical-imaging evaluation, so it does not establish that DINOv2 is effective for medical-imaging tasks. (LLM, conf=0.87)

      Experimental — accuracy not yet measured
  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation
    § B. Experimental Setup · p. 5

    UniverSeg is a UNet-like architecture trained on MegaMedical and designed to generalize to unseen medical segmentation tasks without additional training.

    “UniverSeg : a UNet-like architecture trained on a large collection of medical segmentation datasets (MegaMedical) and designed to generalize to unseen medical segmentation tasks without additional training.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [10] [10] V. I. Butoi, J. J. G. Ortiz, T. Ma, M. R. Sabuncu, J. V. Guttag, and A. V. Dalca, 'Universeg: Universal medical image segmentation.' in ICCV . IEEE, 2023, pp. 21 381-21 394. - source
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § B. Experimental Setup · p. 7

    SAM 2 extends its predecessor by incorporating memory components, enabling adaptation to new tasks and outperforming existing video object segmentation models, especially for temporal tracking.

    “SAM 2 [33] extends its predecessor by incorporating memory components (an encoder, bank, and attention mechanism) enabling adaptation to new tasks and outperforming existing video object segmentation models, especially for temporal tracking.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [33] [33] N. Ravi et al. , 'SAM 2: Segment anything in images and videos,' in The Thirteenth International Conference on Learning Representations , 2025. - source

      The Model section (pp. 4–5) describes SAM 2 as a generalization of SAM with a memory encoder, bank, and attention mechanism. The Introduction (p. 2) and video-segmentation results (p. 10) report that it outperforms prior approaches on video benchmarks. The paper supports memory-enabled segmentation and tracking across frames, but does not establish that those components enable adaptation to new tasks generally; its stated task is promptable visual segmentation. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
  • May combine two claims.

    Premisecitation
    § III. IN-CONTEXT RCA · p. 4

    In-context learning models are computationally efficient and can adapt their predictions based on a small support set.

    “Unlike previous approaches, that rely on training CNNs [29] or computationally expensive atlas-based registration methods as reverse classifiers [5], [28], in-context learning models are computationally efficient and can adapt their predictions based on a small support set.”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [9] [9] X. Wang, X. Zhang, Y. Cao, W. Wang, C. Shen, and T. Huang, 'Seggpt: Towards segmenting everything in context,' in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , 2023, pp. 1130-1140. - source
    • cites [10] [10] V. I. Butoi, J. J. G. Ortiz, T. Ma, M. R. Sabuncu, J. V. Guttag, and A. V. Dalca, 'Universeg: Universal medical image segmentation.' in ICCV . IEEE, 2023, pp. 21 381-21 394. - source
    • cites [11] [11] J. Zhu, Y. Qi, and J. Wu, 'Medical sam 2: Segment medical images as video via segment anything model 2,' CoRR , vol. abs/2408.00874, 2024. - source

      Related Works (p. 3) says few-shot methods adapt to new tasks with minimal annotated samples and describes one-prompt segmentation without retraining. The cited paper does not establish that in-context learning models are computationally efficient, particularly relative to the approaches named in the citing sentence. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
  • May combine two claims.

    Premisecitation
    § B. Performance Interval Estimation · p. 2

    The method in [35] constructs its intervals from nonconformity scores based on softmax scores, which are known to suffer from calibration issues.

    “In addition, the method constructs its intervals from nonconformity scores based on the softmax scores [39], [40], which are known to suffer from calibration issues [41], [42].”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [39] [39] Z. Li, K. Kamnitsas, M. Islam, C. Chen, and B. Glocker, 'Estimating model performance under domain shifts with class-specific confidence scores,' in Medical Image Computing and Computer Assisted Intervention -MICCAI 2022: 25th International Conference, Singapore, September 18-22, 2022, Proceedings, Part VII . Springer-Verlag, 2022, p. 693-703. - source

      The Introduction (p. 2) and Method (p. 4) describe softmax-based confidence scores and explain that uncalibrated probabilities may not reflect real accuracy. The paper describes estimating model performance from those scores, but does not describe constructing intervals from nonconformity scores; that part of the attributed claim is unsupported. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites [40] [40] P. K¨ ohler, J. Fadugba, P. Berens, and L. M. Koch, 'Efficiently correcting patch-based segmentation errors to control image-level performance in retinal images,' in Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning , vol. 250. PMLR, 2024, pp. 841-856. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [41] [41] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, 'On calibration of modern neural networks,' in Proceedings of the 34th International Conference on Machine Learning - Volume 70 . JMLR.org, 2017, p. 1321-1330. - source
  • May combine two claims.

    Premisecitation
    § B. Proposed Method · p. 5

    RCA tends to underestimate segmentation quality, making higher scores more informative.

    “This asymmetry captures the observation that RCA tends to underestimate segmentation quality, making higher scores more informative [5].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [5] [5] V. V. Valindria, I. Lavdas, W. Bai, K. Kamnitsas, E. O. Aboagye, A. G. Rockall, D. Rueckert, and B. Glocker, 'Reverse classification accuracy: Predicting segmentation performance in the absence of ground truth,' IEEE transactions on medical imaging , vol. 36, no. 8, p. 1597-1606, 2017. - source

      In “Quantifying Prediction Accuracy” (p. 5), the paper says that a predicted score consistently lower than the real score could still be useful, but presents this as a hypothetical bias, not an observed tendency. Its results (p. 5) describe one RCA classifier’s predictions as close to the real scores, while the Discussion (p. 8) says RCA can identify high-quality segmentations. The paper therefore supports the usefulness of high scores, but not the claimed general tendency to underestimate quality or the conclusion that high scores are more informative than low ones. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § B. Experimental Setup · p. 7

    A similar retrieval-augmented approach for few-shot medical image segmentation has been explored in [56], demonstrating its effectiveness in similar scenarios.

    “In fact, a similar retrievalaugmented approach for few-shot medical image segmentation has been explored in [56], demonstrating its effectiveness in similar scenarios.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [56] [56] L. Zhao et al. , 'Retrieval-augmented few-shot medical image segmentation with foundation models,' 2024. - source
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § A. Preliminaries: Split Conformal Prediction · p. 4

    CQR maintains the coverage guarantee of conformal prediction while often producing tighter, more locally adaptive intervals compared to residual-based methods, particularly in heteroscedastic settings.

    “This approach maintains the coverage guarantee of CP while often producing tighter, more locally adaptive intervals compared to residual-based methods, particularly in heteroscedastic settings [8].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [8] [8] Y. Romano, E. Patterson, and E. Candes, 'Conformalized quantile regression,' in Advances in Neural Information Processing Systems , vol. 32. Curran Associates, Inc., 2019. - source
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § B. Experimental Setup · p. 6

    UniverSeg was trained on 53 open-access medical segmentation datasets called MegaMedical, covering a wide range of image modalities and structures.

    “It was trained on a collection of 53 open-access medical segmentation datasets called MegaMedical, covering a wide range of image modalities and structures.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [10] [10] V. I. Butoi, J. J. G. Ortiz, T. Ma, M. R. Sabuncu, J. V. Guttag, and A. V. Dalca, 'Universeg: Universal medical image segmentation.' in ICCV . IEEE, 2023, pp. 21 381-21 394. - source
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § C. Evaluation of In-Context RCA · p. 7

    A single-atlas approach offers the strongest results in the classical RCA framework.

    “We compared the performance of In-Context RCA with a singleatlas approach, known to offer the strongest results in the classical RCA framework [5], [28].”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [5] [5] V. V. Valindria, I. Lavdas, W. Bai, K. Kamnitsas, E. O. Aboagye, A. G. Rockall, D. Rueckert, and B. Glocker, 'Reverse classification accuracy: Predicting segmentation performance in the absence of ground truth,' IEEE transactions on medical imaging , vol. 36, no. 8, p. 1597-1606, 2017. - source
    • cites [28] [28] R. Robinson et al. , 'Automatic quality control of cardiac mri segmentation in large-scale population imaging,' in MICCAI 2017 . Springer, 2017, pp. 720-727. - source
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § D. Evaluation of Conformal RCA · p. 8

    PHiSeg is the top performer in the work of [35].

    “In order to enable direct comparison with [35], we adopt PHiSeg [37]-the top performer in that work-as our backbone segmentation model.”
    Quote in the paperNot a substantial claim2 of 2 sources checked
    • cites [35] [35] A. M. Wundram, P. Fischer, M. M¨ uhlebach, L. M. Koch, and C. F. Baumgartner, 'Conformal performance range prediction for segmentation output quality control,' in Uncertainty for Safe Utilization of Machine Learning in Medical Imaging . Springer Nature Switzerland, 2025, pp. 81-91. - source
    • cites [37] [37] C. F. Baumgartner et al. , 'Phiseg: Capturing uncertainty in medical image segmentation,' in Medical Image Computing and Computer Assisted Intervention - MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13-17, 2019, Proceedings, Part II . SpringerVerlag, 2019, p. 119-127. - source
  • Classified as probably not a substantial claim. May combine two claims.

    Findingnone given
    § B. Experimental Setup · p. 6

    Bold values indicate the best performing methods exhibiting significant differences with respect to the non-bold values (p < 0.01, paired test), but no significant differences among themselves.

    “Bold values indicate the best performing methods exhibiting significant differences with respect to the non-bold values ( p &lt; 0 . 01 , paired test), but no significant differences among themselves.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data
    § C. Evaluation of In-Context RCA · p. 7

    Larger reference sets noticeably increase inference time per image without any meaningful improvement in prediction accuracy.

    “Larger sets noticeably increase inference time per image without any meaningful improvement in prediction accuracy.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data
    § C. Evaluation of In-Context RCA · p. 7

    Retrieval-augmentation achieves better results with fewer reference samples than random selection, reducing inference time.

    “However, RA achieves better results with fewer reference samples, what is particularly beneficial during deployment, as it reduces inference time.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data
    § B. Experimental Setup · p. 6

    In both random and retrieval-augmented cases, retrieval-augmentation achieves better performance with fewer samples, as shown by higher correlation and lower MAE.

    “In both cases retrieval-augmentation achieves better performance with fewer samples, as shown by higher correlation and lower MAE (bold indicates the best value).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data
    § Our Contributions : · p. 3

    Experiments on 10 diverse medical imaging datasets demonstrate strong performance across different modalities and anatomy, broadening the scope of RCA while reducing inference time.

    “We conduct extensive experiments on 10 diverse medical imaging datasets, demonstrating strong performance across different modalities and anatomy. We significantly broaden the scope of RCA while reducing inference time.”
    • Also stated in: § ConfIC-RCA: Statistically Grounded Efficient Estimation of Segmentation Quality · p. 1 · boosted
      “Validated across 10 different medical imaging tasks in various organs and modalities, our methods demonstrate robust performance and computational efficiency, offering a promising solution for automated quality control in clinical workflows, where fast and reliable segmentation assessment is essential.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data
    § C. Evaluation of In-Context RCA · p. 7

    As the number of reference samples increases, both random selection and retrieval-augmentation show improved performance, yielding higher correlation and lower mean absolute error.

    “As the number of reference samples increases, both methods show improved performance, yielding higher correlation and lower mean absolute error (MAE).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingreasoning
    § I. INTRODUCTION · p. 2

    Using in-context segmentation models as reverse classifiers significantly reduces the computational cost of RCA while maintaining high accuracy.

    “By using in-context segmentation models as reverse classifiers, our framework significantly reduces the computational cost of RCA while maintaining high accuracy, making it more suitable for clinical use.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data
    § C. Evaluation of In-Context RCA · p. 8

    In-Context RCA demonstrates similar and, in some cases, better results compared to the Atlas method, while being significantly more computationally efficient.

    “We observe that InContext RCA demonstrates similar and, in some cases, better results compared to the Atlas method, while being significantly more computationally efficient.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data
    § A.3 Additional Retrieval-Augmentation Experiments · p. 12

    For 3D-IRCAdB, DINOv2 sometimes outperforms the domain-specific MedImageInsight embeddings, whereas for JSRT, RAD-DINO and MedImageInsight embeddings always outperform DINOv2.

    “For 3D-IRCAdB, DINOv2 sometimes outperforms the domain-specific MedImageInsight embeddings, whereas for JSRT, RAD-DINO and MedImageInsight embeddings always outperform DINOv2, demonstrating the benefit of domain-adapted embeddings in radiological tasks.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data
    § D. Evaluation of Conformal RCA · p. 8

    Compared to the PHiSeg baseline, UniverSeg occasionally produces wider intervals; however, both SAM 2 and Atlas consistently attain lower median widths and reduced spread, indicating more robust uncertainty quantification.

    “Compared to the PHiSeg baseline, UniverSeg occasionally produces wider intervals; however, both SAM 2 and Atlas consistently attain lower median widths and reduced spread, indicating more robust uncertainty quantification.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data
    § D. Evaluation of Conformal RCA · p. 8

    Among Conformal-RCA variants, SAM 2 and Atlas achieve consistently tight intervals, whereas UniverSeg yields somewhat wider intervals.

    “Among our Conformal-RCA variants, SAM 2 and Atlas achieve consistently tight intervals, whereas UniverSeg yields somewhat wider intervals.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data
    § C. Evaluation of In-Context RCA · p. 8

    The classic RCA-atlas method takes on average one minute to process each image, while SAM 2 and UniverSeg achieve the same task in 0.70 seconds and 0.37 seconds respectively, approximately 85 and 162 times faster.

    “Importantly, while the classic RCA-atlas method takes on average one minute to process each image, SAM 2 and UniverSeg achieve the same task in just 0.70 seconds and 0.37 seconds respectively, approximately 85 and 162 times faster.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingnone given
    § D. Evaluation of Conformal RCA · p. 8

    SAM 2 offers a substantial computational speed-up over the atlas-based approach while maintaining similar interval quality.

    “Notably, SAM 2 offers a substantial computational speed-up over the atlas-based approach while maintaining similar interval quality.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data
    § C. Evaluation of In-Context RCA · p. 8

    In the NuCLS histopathology dataset, the SAM 2 variant achieves a correlation of about 0.76 while Atlas methods completely fail.

    “Particularly in highly irregular segmentation problems like the NuCLS histopathology dataset, our SAM 2 variant achieves good correlation of about 0.76 while Atlas methods completely fail.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Premisenone given
    § I. INTRODUCTION · p. 1

    RCA suffers from two main limitations: it is computationally inefficient and lacks a mechanism to quantify uncertainty of its predictions.

    “Even though RCA has been widely adopted in the medical imaging community, it still suffers from two main limitations: it is very inefficient in computational terms and it lacks a mechanism to quantify the uncertainty of its predictions.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning
    § Our Contributions : · p. 3

    ConfIC-RCA is the first model-agnostic framework for segmentation quality estimation that provides prediction intervals with statistical guarantees, enabling image-level uncertainty quantification.

    “We introduce ConfIC-RCA , the first model-agnostic framework for segmentation quality estimation that provides prediction intervals with statistical guarantees, enabling image-level uncertainty quantification.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given
    § B. Experimental Setup · p. 6

    In all cases, 95% confidence intervals were obtained via bootstrapping.

    “In all cases 95% confidence intervals obtained via bootstrapping.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisenone given
    § B. Proposed Method · p. 5

    Traditional RCA yields a point estimate of segmentation quality for an individual image but does not quantify the uncertainty associated with this prediction.

    “Traditional RCA yields a point estimate of segmentation quality for an individual image, as defined in Eq. (2). However, it does not quantify the uncertainty associated with this prediction.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationnone given
    § III. IN-CONTEXT RCA · p. 4

    Dynamic selection of the reference dataset on a per-case basis based on retrieval similarity enables the use of smaller, more relevant reference datasets, improving both computational efficiency and prediction quality.

    “We also introduce a dynamic selection of the reference dataset on a per-case basis based on retrieval similarity, enabling the use of smaller, more relevant reference datasets. This approach improves both computational efficiency and prediction quality.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationnone given
    § Our Contributions : · p. 3

    In-Context RCA is a practical enhancement of traditional RCA that leverages few-shot segmentation models to improve computational efficiency and adapt to new tasks without additional training.

    “We propose In-Context RCA as a practical enhancement of the traditional RCA framework, leveraging few-shot segmentation models to improve computational efficiency and adapt to new tasks without requiring additional training.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationreasoning
    § B. Experimental Setup · p. 7

    Reducing the RCA reference database by dynamically retrieving the most relevant reference images on a per-case basis optimizes computational efficiency while maintaining and even improving the quality of predictions.

    “This allows the model to focus on smaller, more relevant subsets of data, optimizing computational efficiency while maintaining and even improving the quality of predictions.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Premisecitation
    § A. Preliminaries: Split Conformal Prediction · p. 4

    Split conformal prediction provides statistically valid prediction sets or intervals under minimal assumptions.

    “Conformal Prediction (CP) is a model-agnostic, distributionfree framework for uncertainty quantification that provides statistically valid prediction sets or intervals, for classification and regression tasks respectively, under minimal assumptions [6], [7].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [6] [6] V. Vovk, A. Gammerman, and G. Shafer, Algorithmic Learning in a Random World . Springer-Verlag, 2005. - source
    • cites [7] [7] A. N. Angelopoulos and S. Bates, 'A gentle introduction to conformal prediction and distribution-free uncertainty quantification,' 2022. - source
  • May combine two claims.

    Interpretationreasoning
    § B. Proposed Method · p. 5

    Computing empirical quantiles directly from the RCA score distribution captures the uncertainty inherent in the RCA procedure while maintaining computational efficiency.

    “This allows us to naturally capture the uncertainty inherent in the RCA procedure while maintaining computational efficiency.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Preliminaries: Split Conformal Prediction · p. 4

    The only required assumption for split conformal prediction is that calibration samples and test samples are exchangeable (e.g., i.i.d.).

    “The only required assumption is that the calibration samples and the test samples are exchangeable (e.g., the samples are i.i.d.).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] [7] A. N. Angelopoulos and S. Bates, 'A gentle introduction to conformal prediction and distribution-free uncertainty quantification,' 2022. - source
    • The authors state this as an assumption: Calibration samples and test samples are exchangeable (e.g., i.i.d.).
  • May combine two claims.

    Interpretationcitation
    § I. INTRODUCTION · p. 2

    Conformal RCA quantifies uncertainty and provides statistical guarantees that the true segmentation quality lies within the predicted interval with a user-specified probability.

    “Unlike traditional RCA, which provides only a single point estimate (e.g., mean or maximum over reference scores), Conformal RCA quantifies the uncertainty associated with each prediction and provides statistical guarantees that the true segmentation quality lies within the predicted interval with a user-specified probability.”
    • Also stated in: § ConfIC-RCA: Statistically Grounded Efficient Estimation of Segmentation Quality · p. 1 · plain
      “Using tools from split conformal prediction, Conformal RCA produces prediction intervals for segmentation quality providing statistical guarantees that the true score lies within the estimated interval with a user-specified probability.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § B. Proposed Method · p. 5

    The procedure can be applied with any reverse segmenter and can be readily combined with the proposed in-context approach.

    “Note that the procedure can be applied with any reverse segmenter and can thus be readily combined with the proposed in-context approach.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationnone given
    § Our Contributions : · p. 3

    Retrieval-Augmented RCA is a more efficient variant of RCA that uses embedding-based retrieval to select the most relevant reference images as few-shot examples, enhancing efficiency and accuracy.

    “We introduce Retrieval-Augmented RCA, a more efficient variant of RCA that uses embedding-based retrieval to select the most relevant reference images to be used as few-shot examples, enhancing efficiency and accuracy.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § B. Experimental Setup · p. 6

    SAM 2 is a foundational segmentation model originally developed for general-domain image and video segmentation, recently adapted to few-shot medical image segmentation.

    “SAM 2 : a foundational segmentation model originally developed for general-domain image and video segmentation, recently adapted to few-shot medical image segmentation [34].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [34] [34] Y. Bai, Q. Yu, B. Yun, D. Jin, Y. Xia, and Y. Wang, 'Fs-medsam2: Exploring the potential of sam2 for few-shot medical image segmentation without fine-tuning.' CoRR , vol. abs/2409.04298, 2024. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationnone given
    § Our Contributions : · p. 3

    The approach estimates quantiles directly from the RCA reference set instead of training a quantile regression model, avoiding additional training.

    “We base our approach on CQR, but unlike prior work that trains a quantile regression model, we estimate the quantiles directly from the RCA reference set, avoiding additional training.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Premisecitation
    § B. Experimental Setup · p. 7

    For medical few-shot segmentation, SAM 2 treats the support set as initial frames of a sequence with segmentation masks as prompts, effectively transferring segmentation masks from annotated to unannotated examples.

    “For medical few-shot segmentation (FSS) [34], SAM 2 treats the support set as initial frames of a sequence, with their segmentation masks serving as prompts. Despite having no temporal connection between images, this approach effectively transfers segmentation masks from annotated to unannotated examples (see [34] for more details).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [34] [34] Y. Bai, Q. Yu, B. Yun, D. Jin, Y. Xia, and Y. Wang, 'Fs-medsam2: Exploring the potential of sam2 for few-shot medical image segmentation without fine-tuning.' CoRR , vol. abs/2409.04298, 2024. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • The quote only partly states this claim.

    Study design and conductown data
    § D. Evaluation of Conformal RCA · p. 8

    The quantile levels pl=0.3 and ph=0.9 were selected based on a systematic sensitivity analysis over multiple symmetric and asymmetric quantile pairs, evaluating the trade-off between empirical coverage and interval width across datasets.

    “These values were selected based on a systematic sensitivity analysis over multiple symmetric and asymmetric quantile pairs, evaluating the trade-off between empirical coverage and interval width across datasets.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data
    § VI. CONCLUSION · p. 10

    In-Context RCA overcomes the limitations imposed by atlas-based methods, allowing it to tackle scenarios beyond anatomical segmentation, such as cell images.

    “It also overcomes the limitations imposed by atlas-based methods allowing to tackle scenarios beyond anatomical segmentation, such as cell images.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning
    § A.1 Correlation Analysis on New Metrics and Methods · p. 12

    MC Dropout and TTA require additional post-processing to derive an image-level quality proxy.

    “As a result, these methods require additional post-processing to derive an image-level quality proxy.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: TTA and MC Dropout estimate epistemic uncertainty from multiple stochastic segmentation outputs and are commonly used as proxies for segmentation quality.
  • May combine two claims.

    Interpretationnone given
    § B. Performance Interval Estimation · p. 2

    Conformal RCA is model-agnostic and can be used to estimate quality intervals for segmentation masks regardless of the underlying architecture or training procedure used to produce them.

    “In this work, we introduce Conformal RCA, which unlike previous approaches, is model-agnostic and can be used to estimate quality intervals for segmentation masks regardless of the underlying architecture or training procedure used to produce them.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Interpretationnone given
    § VI. CONCLUSION · p. 10

    Conformal RCA relies on the exchangeability assumption between calibration and test sets.

    “A remaining limitation is the reliance of exchangeability between calibration and test sets.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data
    § VI. CONCLUSION · p. 10

    Conformal RCA produces meaningful prediction intervals that outperform existing conformal prediction approaches for segmentation quality control.

    “Conformal RCA demonstrates the ability to produce meaningful prediction intervals that outperform existing conformal prediction approaches for segmentation quality control.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning
    § VI. CONCLUSION · p. 9

    In-Context RCA enables efficient quality estimation using smaller reference databases while either matching or improving prediction accuracy of traditional RCA.

    “This combination enables efficient quality estimation using smaller reference databases while either matching or improving prediction accuracy of traditional RCA.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationnone given
    § D. Evaluation of Conformal RCA · p. 8

    Conformal-RCA is model-agnostic and applicable to any underlying segmentation network regardless of output format.

    “Importantly, Conformal-RCA remains model-agnostic, making it applicable to any underlying segmentation network regardless of output format.”
    • Also stated in: § VI. CONCLUSION · p. 10 · plain
      “Unlike previous interval-estimation approaches that rely on specialized probabilistic architectures and require calibrated softmax outputs, Conformal RCA is fully model-agnostic and easily integrates with any segmentation network.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationreasoning
    § I. INTRODUCTION · p. 1

    The proposed method addresses both limitations of RCA by introducing ConfIC-RCA, a model-agnostic framework that combines two complementary advances.

    “In this work, we address both limitations by introducing ConfIC-RCA (Conformal In-Context RCA) , a model-agnostic framework that combines two complementary advances.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data
    § VI. CONCLUSION · p. 10

    In-Context RCA achieves comparable or superior performance to traditional atlas-based approaches while being up to two orders of magnitude faster.

    “Experimental results conducted across diverse medical imaging modalities, show that In-Context RCA achieves comparable or superior performance to traditional atlas-based approaches while being up to two orders of magnitude faster.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning
    § VI. CONCLUSION · p. 10

    When the exchangeability assumption is violated (e.g., due to domain shifts in scanner settings or patient populations), coverage can degrade, which may affect reliability in real-world clinical deployments.

    “When this assumption is violated (e.g., due to domain shifts in scanner settings or patient populations), coverage can degrade which may affect reliability in real-world clinical deployments.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The calibration and test sets are exchangeable.
54 of 54
  • The paper relies on this without support.

    Limited availability of reference annotations in clinical settings makes continuous assessment of deployed segmentation methods challenging.

    “Once a segmentation method is deployed in practice, continuously assessing its real performance on new unseen data samples becomes a challenging task, mainly due to the limited availability of reference annotations in clinical settings.”
  • The authors state this as an assumption.

    Calibration samples and test samples are exchangeable (e.g., i.i.d.).

    “The only required assumption is that the calibration samples and the test samples are exchangeable (e.g., the samples are i.i.d.).”
    “Assuming exchangeability between the test sample and calibration set, the constructed interval provides the marginal coverage guarantee stated in Theorem 3, as shown in [8].”
  • The authors state this as an assumption.

    Higher values of the quality metric ρ correspond to better quality estimations.

    “assuming that higher values of ρ correspond to better quality estimations.”
  • The authors state this as an assumption.

    If the predicted segmentation S_I is of high quality, the RCA segmenter should perform well on at least some reference images, and if S_I is of poor quality, it should perform poorly on the reference database.

    “The underlying hypothesis is that if S I is of high quality, the RCA segmenter should perform well on at least some of the reference images, resulting in high values of ρ . Conversely, if S I is of poor quality, the segmenter is expected to perform poorly on the reference database.”
  • The authors state this as an assumption.

    The calibration and test sets are exchangeable.

    “A remaining limitation is the reliance of exchangeability between calibration and test sets.”
  • The paper relies on this without support.

    TTA and MC Dropout estimate epistemic uncertainty from multiple stochastic segmentation outputs and are commonly used as proxies for segmentation quality.

    “These methods estimate epistemic uncertainty from multiple stochastic segmentation outputs and are commonly used as proxies for segmentation quality.”
    “In contrast, approaches like MC Dropout and TTA estimate epistemic uncertainty rather than segmentation quality itself.”
6 of 6

Cites

Flagged first, then by citations

Research works printed but not linked: 5

  1. [30][30] C. Ouyang, C. Biffi, C. Chen, T. Kart, H. Qiu, and D. Rueckert, 'Selfsupervised learning for few-shot medical image segmentation,' IEEE Transactions on Medical Imaging , vol. 41, no. 7, pp. 1837-1848, 2022. -
  2. [31][31] J. Snell, K. Swersky, and R. Zemel, 'Prototypical networks for fewshot learning,' in Proceedings of the 31st International Conference on Neural Information Processing Systems . Curran Associates Inc., 2017, p. 4080-4090. -
  3. [34][34] Y. Bai, Q. Yu, B. Yun, D. Jin, Y. Xia, and Y. Wang, 'Fs-medsam2: Exploring the potential of sam2 for few-shot medical image segmentation without fine-tuning.' CoRR , vol. abs/2409.04298, 2024. - · arXiv
  4. [40][40] P. K¨ ohler, J. Fadugba, P. Berens, and L. M. Koch, 'Efficiently correcting patch-based segmentation errors to control image-level performance in retinal images,' in Proceedings of The 7nd International Conference on Medical Imaging with Deep Learning , vol. 250. PMLR, 2024, pp. 841-856. -
  5. [51][51] L. Soler et al. , '3d image reconstruction for comparison of algorithm database: A patient specific anatomical and medical image database,' IRCAD, Strasbourg, France, Tech. Rep , vol. 1, no. 1, 2010. -

Cited by

Sorted by citations

Authors
Cosarinsky, M., Billot, R., Mansilla, L., Jimenez, G., Gaggión, N., Fu, G., Tirer, T., & Ferrante, E.
Affiliations
  • National University of General San Martín (AR)
  • Consejo Nacional de Investigaciones Científicas y Técnicas (AR)
  • Centre National de la Recherche Scientifique (FR)
  • Bar-Ilan University (IL)
Venue
arXiv
Publisher
arXiv
Published
2025
Volume
45
Issue
7
Pages
3545-3557
ISSN
0278-0062 (ISSN-L)
OpenAlex
W7147452949
Cited by
0
References
65
FWCI
0.00
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Medical Imaging (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Funders
Israel Science Foundation (1940/23)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
automated image segmentation, conformal prediction, split conformal prediction, prediction intervals, in-context learning, automated quality control, medical image segmentation
MeSH terms
Machine Learning, Classification Algorithms, Algorithms, Humans, Image Processing, Computer-Assisted

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-21.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-21.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review