The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index; 7 of the 22 sources its claims cite were checked for support.
Reverse Classification Accuracy: Predicting Segmentation Performance in the Absence of Ground Truth
Open accessAbstract
When integrating computational tools, such as automatic segmentation, into clinical practice, it is of utmost importance to be able to assess the level of accuracy on new data and, in particular, to detect when an automatic method fails. However, this is difficult to achieve due to the absence of ground truth. Segmentation accuracy on clinical data might be different from what is found through cross validation, because validation data are often used during incremental method development, which can lead to overfitting and unrealistic performance expectations. Before deployment, performance is quantified using different metrics, for which the predicted segmentation is compared with a reference segmentation, often obtained manually by an expert. But little is known about the real performance after deployment when a reference is unavailable. In this paper, we introduce the concept of reverse classification accuracy (RCA) as a framework for predicting the performance of a segmentation method on new data. In RCA, we take the predicted segmentation from a new image to train a reverse classifier, which is evaluated on a set of reference images with available ground truth. The hypothesis is that if the predicted segmentation is of good quality, then the reverse classifier will perform well on at least some of the reference images. We validate our approach on multi-organ segmentation with different classifiers and segmentation methods. Our results indicate that it is indeed possible to predict the quality of individual segmentations, in the absence of ground truth. Thus, RCA is ideal for integration into automatic processing pipelines in clinical routine and as a part of large-scale image analysis studies.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper introduces reverse classification accuracy (RCA), a framework that predicts per-case segmentation quality without ground truth by training a classifier on a predicted segmentation as pseudo-GT and evaluating it on reference images with known GT. Across multi-organ MRI experiments, RCA with Single-Atlas label propagation best predicted Dice scores, while distance-based metrics and constrained CNNs were less reliable.
Key findings
- RCA can predict individual segmentation accuracy without ground truth. Best results used Single-Atlas as the RCA classifier: correlations of 0.955, 0.973, and 0.962 for Random Forests, CNNs, and Multi-Atlas segmentation methods, with MAEs of 0.051, 0.052, and 0.067 and 3-category accuracies of 0.888, 0.811, and 0.822.
- Atlas Forests also performed well: correlations of 0.881, 0.828, and 0.863, MAEs of 0.120, 0.166, and 0.168, and 3-category accuracies of 0.783, 0.623, and 0.749 for the same three segmentation methods. Constrained CNNs were least suitable, with correlations below 0.78 and MAEs above 0.2.
- In a failure-detection experiment using degraded Random Forests, Atlas Forests achieved correlation 0.853, MAE 0.096, and accuracy 0.884; Single-Atlas achieved correlation 0.875, MAE 0.097, and accuracy 0.928; constrained CNNs had correlation 0.237, MAE 0.139, and accuracy 0.924. The bladder was an exception where failed segmentations were not reliably detected.
- RCA predicted overlap-based metrics better than distance-based metrics: DSC correlation 0.881, MAE 0.120, accuracy 0.783; Jaccard index correlation 0.899, MAE 0.110, accuracy 0.749; Hausdorff distance correlation 0.189, MAE 41.38, accuracy 0.387; average surface distance correlation 0.022, MAE 4.120, accuracy 0.344; relative volume difference correlation 0.474, MAE 0.051, accuracy 0.678.
Methods
- Whole-body multi-sequence MRI (T1w Dixon and T2w) from 35 healthy volunteers in the MALIBO study, with manual segmentations of 15 anatomical structures, yielding 525 structures (15 organs × 35 subjects). Evaluation used 3-fold cross-validation.
- Three segmentation methods were tested: Random Forests, 3D CNNs (DeepMedic), and probabilistic multi-atlas label propagation. Three RCA classifiers were tested: Atlas Forests, constrained CNNs, and Single-Atlas label propagation, giving nine combinations.
- Prediction accuracy was quantified by correlation between predicted and real Dice similarity coefficient (DSC), mean absolute error (MAE), and 3-category classification accuracy using DSC ranges [0.0, 0.6) for bad, [0.6, 0.8) for medium, and [0.8, 1.0] for good. The RCA proxy was the maximum DSC across reference images. A leave-one-subject-out Random Forest regression calibration experiment was also run.
Limitations
- The paper states that RCA assumes a linear relationship between predicted and real scores close to identity, which was only found with Single-Atlas; off-diagonal correlation for Atlas Forests may require calibration, but calibration risks overfitting. It also states that distance-based metrics such as Hausdorff distance and average surface distance cannot be accurately predicted with the current setting, and that constrained CNNs trained on single images only work for major organs and struggle with small structures, producing many zero predictions.
- Evident from the text: validation used 35 healthy volunteers rather than clinical or pathological data, and RCA requires a reference database with available ground truth. The 3-category DSC thresholds are described as somewhat arbitrary and would need adjustment for each application.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
7 of 22 cited sources checked · 3 supported · 0 flagged · accuracy not yet measured
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, these methods cannot be applied to a more general task, such as an image with many different class labels to be segmented.”
What the sources say
- Partly supportedcites [18] [18] P. L. Correia and F. Pereira, 'Stand-alone objective segmentation quality evaluation,' EURASIP Journal on Applied Signal Processing , vol. 2002, no. 1, pp. 389-400, 2002. - sourceExperimental — accuracy not yet measured
The Introduction and Section 3.2 say that stand-alone evaluation works best in constrained scenarios and that a single general-purpose composite metric cannot be established. They do not identify a large number of class labels as the limitation; Section 3.2 also says metrics can be adapted for content outside its two proposed classes. The categorical claim that these methods cannot be applied to such an image therefore goes beyond the paper. (LLM, conf=0.86)
- Not checked — no full text of the sourcecites [19] [19] F. Ge, S. Wang, and T. Liu, 'New benchmark for image segmentation evaluation,' Journal of Electronic Imaging , vol. 16, no. 3, pp. 033 011033 011, 2007. - source
- Not checked — no full text of the sourcecites [20] [20] L. Goldmann, T. Adamek, P. Vajda, M. Karaman, R. M¨ orzinger, E. Galmar, T. Sikora, N. E. O?Connor, T. Ha-Minh, T. Ebrahimi et al. , 'Towards fully automatic image segmentation evaluation,' in International Conference on Advanced Concepts for Intelligent Vision Systems . Springer, 2008, pp. 566-577. - source
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“However, the approaches [31], [33] can only be applied when a training database with good and bad segmentations is available from which a mapping from features to segmentation accuracy can be learned.”
What the sources say
- Supportedcites [31] [31] T. Kohlberger, V. Singh, C. Alvino, C. Bahlmann, and L. Grady, 'Evaluating segmentation error without ground truth,' in International Conference on Medical Image Computing and Computer-Assisted Intervention . Springer, 2012, pp. 528-536. - source
The Introduction describes training a regression model to predict segmentation error from features using errors measured against known ground truth. The Experiments section also evaluates its ability to distinguish acceptable from non-acceptable segmentations. Ground truth is needed for training, not for applying the trained model. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [33] [33] K. Frounchi, L. C. Briand, L. Grady, Y. Labiche, and R. Subramanyan, 'Automating image segmentation verification and validation by learning test oracles,' Information and Software Technology , vol. 53, no. 12, pp. 1337-1348, 2011. - source
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“An appealing property of the proposed framework is that unlike the supervised methods in [33] and [31] no training data is required that captures examples of good and bad segmentations.”
What the sources say
- Not checked — no full text of the sourcecites [33] [33] K. Frounchi, L. C. Briand, L. Grady, Y. Labiche, and R. Subramanyan, 'Automating image segmentation verification and validation by learning test oracles,' Information and Software Technology , vol. 53, no. 12, pp. 1337-1348, 2011. - source
- Supportedcites [31] [31] T. Kohlberger, V. Singh, C. Alvino, C. Bahlmann, and L. Grady, 'Evaluating segmentation error without ground truth,' in International Conference on Medical Image Computing and Computer-Assisted Intervention . Springer, 2012, pp. 528-536. - source
The Introduction and Method describe training a regressor on segmentation errors measured against known ground truth. This supports the cited paper’s role as an example of a supervised method requiring training data; the comparison with RCA is the citing authors’ own claim. (LLM, conf=0.92)
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“While the idea of RCA is similar to reverse validation [34] and reverse testing [35], the important difference is that in our approach we train a reverse classifier on every single instance while the approaches in [34], [35] train single classifiers over the whole test set and its predictions jointly to find out what the best original predictor is.”
What the sources say
- Supportedcites [34] [34] E. Zhong, W. Fan, Q. Yang, O. Verscheure, and J. Ren, 'Cross validation framework to choose amongst models and datasets for transfer learning,' in Joint European Conference on Machine Learning and Knowledge Discovery in Databases . Springer, 2010, pp. 547-562. - source
Section 3.1 (Reverse Validation; page not provided) describes predicting labels for the target instances, combining them into a training set, and training a reverse model from that set. The reverse model is trained jointly on the instances rather than separately for each instance; the procedure repeats for each fold. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [35] [35] W. Fan and I. Davidson, 'Reverse testing: an efficient framework to select amongst classifiers under sample selection bias,' in Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . ACM, 2006, pp. 147-156. - source
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our results indicate that it is indeed possible to predict the quality of individual segmentations, in the absence of ground truth.”
- Also stated in: § B. Contribution · p. 2 · hedged
“Our results indicate that, at least to some extent, it is indeed possible to predict the performance level of a segmentation method on each individual case, in the absence of ground truth.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- The authors state this as an assumption: A suitable reference database containing multiple segmented images that capture expected variability is available.
- The authors state this as an assumption: The RCA classifier works best on images that are somewhat similar to the image I.
- The authors state this as an assumption: There is a correlation between the segmentation metric values computed on reference images and the values one would get for the new image if ground truth were available.
- The authors state this as an assumption: The maximum DSC score over all reference images correlates well with the real DSC on the new image if ground truth were available.
- The authors state this as an assumption: The RCA classifier performs well on at least some reference images if and only if the predicted segmentation S_I is of good quality, and performs poorly on all reference images if S_I is of bad quality.
- Also stated in: § B. Contribution · p. 2 · hedged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We observe that the zero predictions have mostly an impact on CNNs, either employed as original segmentation method or as RCA classifier.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The single-atlas label propagation makes accurate predictions of segmentation quality across all 15 anatomical structures.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and good 3-category classification accuracies between 81% and 89%”
- Also stated in: § C. Results for Predicting Dice's Similarity Coefficients · p. 6 · plain
“BEST RESULTS ARE OBTAINED WITH THE SINGLE-ATLAS APPROACH YIELDING THE LOWEST ERROR BETWEEN REAL AND PREDICTED DSC SCORES AND HIGH CLASSIFICATION ACCURACY.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Also stated in: § C. Results for Predicting Dice's Similarity Coefficients · p. 6 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is visually confirmed by the scatter plots in the right column of Fig. 2 which show good linear relation close to the diagonal between predicted and real scores for most structures in the case where Random Forests or Multi-Atlas are used as the original segmentation method.”
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Some of the original segmentation methods have problems segmenting structures such as the adrenal gland and clavicles.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Probably not a substantial claim. 19
- It may state two separate claims as one. 21
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“While unsupervised methods can be applied to scenarios where the main purpose of segmentation is to yield visually consistent results that are meaningful to a human observer, the application in medical settings is unclear.”
What the sources say
- Could not verifycites [24] [24] S. Chabrier, B. Emile, C. Rosenberger, and H. Laurent, 'Unsupervised performance evaluation of image segmentation,' EURASIP Journal on Applied Signal Processing , vol. 2006, pp. 217-217, 2006. - sourceExperimental — accuracy not yet measured
The Introduction says supervised evaluation criteria are widely used in medical applications, while Section 2 describes unsupervised criteria in relation to human perception. The paper’s real-image examples are aerial and radar images (Section 4). None of these passages addresses whether unsupervised methods can be applied in medical settings or says that their application there is unclear. (LLM, conf=0.76)
- Not checked — no full text of the sourcecites [25] [25] H. Zhang, J. E. Fritts, and S. A. Goldman, 'Image segmentation evaluation: A survey of unsupervised methods,' Computer Vision and Image Understanding , vol. 110, no. 2, pp. 260-280, 2008. - source
Strength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The Dice's similarity coefficient is the most widely used measure for evaluating segmentation performance 5 , and in our main results we focus on evaluating how well DSC can be predicted using our RCA framework.”
What the sources say
- Partly supportedcites [11] [11] E. Konukoglu, B. Glocker, D. H. Ye, A. Criminisi, and K. M. Pohl, 'Discriminative segmentation-based evaluation through shape dissimilarity,' IEEE Transactions on Medical Imaging , vol. 31, no. 12, pp. 2278-2289, 2012. - sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) describes Dice’s similarity coefficient as an example of a popular, commonly used segmentation-based score. Section II (page unavailable) lists it among several popular measures, but does not establish that it is the most widely used. (LLM, conf=0.85)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The basic idea of reverse validation [34] is based on reverse testing [35], where a new classifier is trained on predictions on the test data and evaluated again on the training data.”
What the sources say
- Partly supportedcites [34] [34] E. Zhong, W. Fan, Q. Yang, O. Verscheure, and J. Ren, 'Cross validation framework to choose amongst models and datasets for transfer learning,' in Joint European Conference on Machine Learning and Knowledge Discovery in Databases . Springer, 2010, pp. 547-562. - sourceExperimental — accuracy not yet measured
Section 3.1 (page unavailable) describes training a new model on predicted labels for target-domain data, then using it to classify held-out source-domain data. But the paper calls this reverse validation. Section 6 (page unavailable) identifies reverse testing as a different technique from Fan and Davidson (2006); it does not establish that reverse validation is based on it. (LLM, conf=0.83)
- Not checked — no full text of the sourcecites [35] [35] W. Fan and I. Davidson, 'Reverse testing: an efficient framework to select amongst classifiers under sample selection bias,' in Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . ACM, 2006, pp. 147-156. - source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The constrained CNNs seem to only work for predicting segmentation accuracy in case of major organs such as liver, lungs, and the spine but clearly struggle with smaller structures leading to many zero predictions even when the real DSC is rather high.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Both the MAEs and classification accuracies improve significantly for the case of Atlas Forests and constrained CNNs.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Atlas Forests (left) and single-atlas label propagation (right) make highly accurate predictions in the low DSC ranges and thus, are able to correctly detect such failed segmentations, with the exception of the bladder.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Prediction accuracy for precision is lower than for recall.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Atlas Forests and Single-Atlas, however, result in high correlations, low MAEs and very good classification accuracies.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the case of the constrained CNNs, we observe that the prediction quality is lowest confirmed by the scatter plots and all quantitative measures, with correlations below 0.78 and MAEs above 0.2.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Low real DSC scores are correctly predicted and failed segmentations are identified.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“From the scatter plots in Fig. 2 we can see that all three segmentation methods perform reasonably well on most major organs with no failure cases among structures such as liver, heart, and lungs.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We have also experimented with a constrained CNN trained on single images, which only works well for major organs such as liver, lungs and spine.”
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Both, Atlas Forests and constrained CNNs work best for larger organs such as liver, lungs, and pelvis while leading to many zero predictions for smaller structures such as adrenal gland and clavicles.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The constrained CNNs are again suffering from many zero predictions and less suitable for making accurate predictions.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Good correlation is obtained between predicted and real overlap based scores, with low MAEs, and high accuracies.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For Atlas Forests and Single-Atlas the effect on the accuracies is very little, confirming that those both are well suited within the RCA framework, independent of the original segmentation method.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The predictions for RVD are good, with high classification accuracy of 0.68%, similar to the overlap based scores.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Second, the clinical data might be different due to varying imaging protocols or artefacts caused by pathology.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we introduce the concept of reverse classification accuracy (RCA) as a framework for predicting the performance of a segmentation method on new data.”
- Also stated in: § B. Contribution · p. 2 · plain
“The main contribution of this paper is the introduction of the concept of reverse classification accuracy (RCA) to assess the segmentation quality of an individual image in the absence of GT.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- The authors state this as an assumption: A suitable reference database containing multiple segmented images that capture expected variability is available.
- The authors state this as an assumption: The RCA classifier works best on images that are somewhat similar to the image I.
- The authors state this as an assumption: There is a correlation between the segmentation metric values computed on reference images and the values one would get for the new image if ground truth were available.
- The authors state this as an assumption: The maximum DSC score over all reference images correlates well with the real DSC on the new image if ground truth were available.
- The authors state this as an assumption: The RCA classifier performs well on at least some reference images if and only if the predicted segmentation S_I is of good quality, and performs poorly on all reference images if S_I is of bad quality.
- Also stated in: § B. Contribution · p. 2 · plain
In the paper
“Another work by [30] has quantitatively evaluated the performance of several segmentation algorithms by regioncorrelation matrix. The limitation of this work is that it cannot evaluate the segmentation performance of a particular method on a particular image independently.”
What the sources say
- Not checked — no full text of the sourcecites [30] [30] K. Sikka and T. M. Deserno, 'Comparison of algorithms for ultrasound image segmentation without ground truth,' in SPIE Medical Imaging . International Society for Optics and Photonics, 2010, pp. 76 271C76 271C. - source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Detailed manual segmentations of 15 anatomical structures, including abdominal organs (heart, left/right lung, liver, adrenal gland, gall bladder, left/right kidney, spleen, pancreas, bladder) and bones (spine, left/right clavicle, pelvis) have been generated by clinical experts as part of the study. These manual segmentations will serve as GT in the quantitative evaluation.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is most likely caused by the difficulty of training the CNNs with single images and small structures which does not provide sufficient amounts of training data.”
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Note, that the mean or median scores are not very useful measures as we do not expect the RCA classifier to work well on the majority of the reference images.”
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: The RCA classifier works best on images that are somewhat similar to the image I.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Afterall, the RCA classifier does overfit to the single image and will not generalize to perform well on dissimilar images.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
- The authors state this as an assumption: The RCA classifier works best on images that are somewhat similar to the image I.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With less number of filters, the number of parameters is substantially decreased, which leads to faster computations, but more importantly, reduces overfitting when trained on a single image.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In conclusion, it seems RCA works very well for overlap based measures and for RVD to some extent, while distance based metrics cannot be accurately predicted with the current setting and would require further investigation.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“Since they operate as voxel-wise classifiers, they do not (necessarily) require preregistration of the images neither at training nor testing time.”
What the sources say
- Not checked — no full text of the sourcecites [36] [36] D. Zikic, B. Glocker, and A. Criminisi, 'Encoding atlases by randomized classification forests for efficient multi-atlas label propagation,' Medical Image Analysis , vol. 18, no. 8, pp. 1262-1273, 2014. - source
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Calibration, however, comes with a risk of overfitting as the method will learn the relationship on the available training data but might not generalize to new data.”
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For the purpose of RCA, however, there is only a single atlas and thus no label fusion is required.”
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Because the real DSC for these is zero with no voxels labelled in the segmentation map, the RCA predictions are always correct as there are no labels for the RCA classifier for this structure.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By reducing the feature maps without changing the architecture in terms of number of layers the network preserves its capability to see large image context as the size of the receptive field remains unchanged.”
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Note, that those ranges are somewhat arbitrary, in particular, as the quality of absolute DSC values is highly depending on the structure of interest. So in practice, those ranges would need to be adjusted specifically to the application at hand.”
Strength:hedged· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“The two metrics capture different parts of segmentation error; under-segmentation is not reflected in precision, while over-segmentation is not captured in recall”
What the sources say
- Not checked — no full text of the sourcecites [7] [7] W. R. Crum, O. Camara, and D. L. Hill, 'Generalized overlap measures for evaluation and validation in medical image analysis,' IEEE Transactions on Medical Imaging , vol. 25, no. 11, pp. 1451-1461, 2006. - source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Such an approach cannot not be used to directly predict segmentation scores such as DSC, but can be useful to inform automatic quality control or to automatically select the best segmentation from a set of candidates.”
What the sources say
- Not checked — no full text of the sourcecites [32] [32] L. Grady, V. Singh, T. Kohlberger, C. Alvino, and C. Bahlmann, 'Automatic segmentation of unknown objects, with application to baggage security,' in European Conference on Computer Vision (ECCV) . Springer, 2012, pp. 430-444. - source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“RCA has the advantage of allowing to predict the accuracy for each individual case, while at the same time aggregating over such accuracy predictions allows drawing conclusions for the overall performance of a particular segmentation method.”
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- The authors state this as an assumption: A suitable reference database containing multiple segmented images that capture expected variability is available.
- The authors state this as an assumption: The RCA classifier works best on images that are somewhat similar to the image I.
- The authors state this as an assumption: There is a correlation between the segmentation metric values computed on reference images and the values one would get for the new image if ground truth were available.
- The authors state this as an assumption: The maximum DSC score over all reference images correlates well with the real DSC on the new image if ground truth were available.
- The authors state this as an assumption: The RCA classifier performs well on at least some reference images if and only if the predicted segmentation S_I is of good quality, and performs poorly on all reference images if S_I is of bad quality.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As the RCA framework is generic, other methods can be considered and it might be necessary to select the most appropriate one for the application at hand.”
Strength:hedged· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The authors state this as an assumption.
A suitable reference database containing multiple segmented images that capture expected variability is available.
“we further assume that a suitable reference database is available that contains multiple segmented images (or atlases) T = { ( J k , S GT J k ) } m k =1 that capture the expected variability.”
“The assumption is that in machine learning approaches, such a database is usually already available, but it could also be specifically constructed for the purpose of RCA.”
The authors state this as an assumption.
The RCA classifier works best on images that are somewhat similar to the image I.
“In fact, the RCA classifier f I,S I is assumed to work best on images that are somewhat similar to I .”
The authors state this as an assumption.
There is a correlation between the segmentation metric values computed on reference images and the values one would get for the new image if ground truth were available.
“The underlying hypothesis in our RCA framework is that there is a correlation between the values computed on ( S J , S GT J ) and the values one would get for the pair ( S I , S GT I ) , where S GT I is the reference GT of image I which in practice, however, is unavailable.”
The authors state this as an assumption.
The maximum DSC score over all reference images correlates well with the real DSC on the new image if ground truth were available.
“It is expected that the maximum DSC score over all reference images correlates well with the real DSC that one would get on the new image if GT were available.”
The authors state this as an assumption.
The RCA classifier performs well on at least some reference images if and only if the predicted segmentation S_I is of good quality, and performs poorly on all reference images if S_I is of bad quality.
“When testing the RCA classifier on all of the available m reference images, we expect that the RCA classifier performs well on at least some of these, if and only if the predicted segmentation S I is of good quality. If S I is of bad quality, we expect the RCA classifier to perform poorly on all reference images.”
The authors state this as an assumption.
There is a linear relationship between predicted and real scores which should be close to an identity mapping.
“The drawback, however, is that we assume a linear relationship between predicted and real scores which should be close to an identity mapping”
The paper relies on this without support.
DSC is the most widely considered segmentation evaluation measure.
“Still, DSC is the most widely considered measure”
The paper relies on this without support.
Manual quality control by visual inspection is not feasible when thousands of images are automatically processed for the purpose of deriving population statistics.
“In settings where thousands of images are automatically processed for the purpose of deriving population statistics, it is not feasible to employ manual quality control with visually inspection of the segmentation results.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The distribution of scores over the reference set serves as a proxy for the quality of the original segmentation.”
From ConfIC-RCA: Statistically Grounded Efficient Estimation of Segmentation Quality (2025) · cites this paper as [5]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This asymmetry captures the observation that RCA tends to underestimate segmentation quality, making higher scores more informative [5].”
From ConfIC-RCA: Statistically Grounded Efficient Estimation of Segmentation Quality (2025) · cites this paper as [5]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Reverse Classification Accuracy (RCA) [5] is a framework originally designed to predict the performance of an image segmentation method in scenarios where GT masks are unavailable.”
From ConfIC-RCA: Statistically Grounded Efficient Estimation of Segmentation Quality (2025) · cites this paper as [5]
The cited source says what the claim attributes to it.
In the citing paper
“It works by using the predicted segmentation of a new image as a pseudo-ground truth to train a reverse classifier (a segmentation network in our case) 1 . This model is then applied to a set of reference images with known GT, and its performance is evaluated using standard metrics like the DSC.”
From ConfIC-RCA: Statistically Grounded Efficient Estimation of Segmentation Quality (2025) · cites this paper as [5]
The cited source says what the claim attributes to it.
In the citing paper
“Unlike previous studies that explored RCA focusing on cardiac MRI [28], [29] or multi-organ segmentation on whole-body MRI [5] only, we broaden our analysis to a wider range of medical imaging challenges.”
From ConfIC-RCA: Statistically Grounded Efficient Estimation of Segmentation Quality (2025) · cites this paper as [5]
The cited source says what the claim attributes to it.
In the citing paper
“We compared the performance of In-Context RCA with a singleatlas approach, known to offer the strongest results in the classical RCA framework [5], [28].”
From ConfIC-RCA: Statistically Grounded Efficient Estimation of Segmentation Quality (2025) · cites this paper as [5]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The HAM10000 dataset [34] contains 2500 dermatoscopic images of pigmented skin lesions.”
From Uncertainty-aware segmentation quality prediction via deep learning Bayesian Modeling: Comprehensive evaluation and interpretation on skin cancer and liver segmentation (2025) · cites this paper as [34]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
RVD classification accuracy is misstated as 0.68%
MinorTable III gives the RVD classification accuracy as 0.678, or 67.8%, rather than approximately 0.68%.
Reported 0.68 · recomputed 67.8
“The predictions for RVD are good, with high classification accuracy of 0.68%, similar to the overlap based scores.”
Atlas Forests classification accuracies are reported as fractions of a percent
MinorThe cited table reports accuracies of 0.623, 0.749, and 0.783, corresponding to approximately 62%, 75%, and 78%, not 0.62%, 0.75%, and 0.78%.
Reported 0.62 · recomputed 62.3
“classification accuracy going down to 0.62%, 0.75% and 0.78% depending on the original segmentation method.”
Calibration does not improve every Atlas Forests classification accuracy
MinorThe text says classification accuracies improve with calibration for Atlas Forests, but Table IV shows the Multi-Atlas/Atlas Forests accuracy falling from 0.749 to 0.745.
“Both the MAEs and classification accuracies improve significantly for the case of Atlas Forests and constrained CNNs.”
“| Multi-Atlas | Atlas Forests | 0.863 | 0.939 | 0.168 | 0.065 | 0.749 | 0.745 |”
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Sudlow, C., Gallacher, J., Allen, N., Beral, V., Burton, P., Danesh, J., Downey, P., Elliott, P., Green, J., Landray, M., et al. (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001779 · Cited by 12,728
Measures of the Amount of Ecologic Association Between Species
Dice, L. R. (1945) · Ecology · doi:10.2307/1932409 · Cited by 12,187
Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation
Kamnitsas, K., Ledig, C., Newcombe, V. F. J., Simpson, J. P., Kane, A. D., Menon, D. K., Rueckert, D., & Glocker, B. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.10.004 · Cited by 3,498 · cited 3×
“In this application, we evaluate the prediction accuracy of RCA in the context of three different state-of-the-art segmentation methods, a Random Forest approach [4], a deep learning approach using 3D CNNs [39], and a probabilistic multi-atlas label propagation approach [40].”
BackgroundMethodInfluentialSelf-citationMetrics for evaluating 3D medical image segmentation: analysis, selection, and tool
Taha, A. A., & Hanbury, A. (2015) · BMC Medical Imaging · doi:10.1186/s12880-015-0068-x · Cited by 3,247
Warfield, S. K., Zou, K. H., & Wells, W. M. (2004) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2004.828354 · Cited by 2,072
Graph Cuts and Efficient N-D Image Segmentation
Boykov, Y., & Funka-Lea, G. (2006) · International Journal of Computer Vision · doi:10.1007/s11263-006-7934-5 · Cited by 1,910
Statistical validation of image segmentation quality based on a spatial overlap index1
Zou, K. H., Warfield, S. K., Bharatha, A., Tempany, C. M., Kaus, M. R., Haker, S. J., Wells, W. M., Jolesz, F. A., & Kikinis, R. (2004) · Academic Radiology · doi:10.1016/s1076-6332(03)00671-8 · Cited by 1,860
Statistical shape models for 3D medical image segmentation: A review
Heimann, T., & Meinzer, H.-P. (2009) · Medical Image Analysis · doi:10.1016/j.media.2009.05.004 · Cited by 1,462
Image segmentation evaluation: A survey of unsupervised methods
Zhang, H., Fritts, J. E., & Goldman, S. A. (2008) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2007.08.003 · Cited by 963
Flagged first, then by citations
Cited by
Sorted by citations
Items 101–154 of 154
Deep Quality Estimation: Creating Surrogate Models for Human Quality Ratings
Kofler, F., Ezhov, I., Fidon, L., et al. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-33842-7_1 · Cited by 3
Case-control comparison brain lesion segmentation for early infarct detection
Ting, F. F., Sim, K. S., & Lim, C. P. (2018) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2018.08.011 · Cited by 3
A Volumetric Convolutional Neural Network for Brain Tumor SegmentationPreprint
Sherman, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.02654 · Cited by 3
Robust Image Segmentation Quality Assessment without Ground TruthPreprint
Zhou, L., Deng, W., & Wu, X. (2019) · arXiv (Cornell University) · Cited by 3
INFORMER- Interpretability Founded Monitoring of Medical Image Deep Learning Models
Shu, S. Z., de Mortanges, A. P., Poellinger, A., Mahapatra, D., & Reyes, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73158-7_20 · Cited by 2
Hamghalam, M., Peoples, J., Kobayashi, K. S. M., Park, G. S. H., Kwak, E., Bunker, E. C., Gangai, N. C., Gonen, M., Chun, Y. S., Kang, H. C., et al. (2026) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2026.109233 · Cited by 2
Pusparani, Y., Lin, C.-Y., Jan, Y., Liau, B.-Y., Lin, F., Furqon, E. N., Talal, M., Pravin, S. C., Tsai, Z.-R., & Lung, C.-W. (2026) · Scientific Reports · doi:10.1038/s41598-026-38220-4 · Cited by 3
Zeng, D., Ding, Y., Yuan, H., et al. (2021) · Proceedings - ACM IEEE Design Automation Conference · doi:10.1109/dac18074.2021.9586158 · Cited by 2
Key techniques for classification of thorax diseases based on deep learning
Zhao, G., Shao, S., & Yu, M. (2022) · International Journal of Imaging Systems and Technology · doi:10.1002/ima.22773 · Cited by 2
Quality Assurance based on Deep Learning for Pelvic OARs Delineation in Radiotherapy
Yu, H., He, Y., Fu, Y., Li, X., Zhang, J., & Liu, H. (2022) · Current Medical Imaging Formerly Current Medical Imaging Reviews · doi:10.2174/1573405618666220621121225 · Cited by 2
CheXmask: a large-scale dataset of anatomical segmentation masks for multi-center chest x-ray images
Gaggion, N., Mosquera, C., Mansilla, L., Saidman, J. M., Aineseder, M., Milone, D. H., & Ferrante, E. (2023) · JAIIO, Jornadas Argentinas de Informática · doi:10.48550/arxiv.2307.03293 · Cited by 4
Artificial Intelligence in the Image-Guided Care of Atrial Fibrillation
Lyu, Y., Bennamoun, M., Sharif, N., Lip, G. Y. H., & Dwivedi, G. (2023) · Life · doi:10.3390/life13091870 · Cited by 3
A Machine Learning Approach To Improve Consistency In User-Driven Medical Image Analysis
Eriksen, E. R. W. (2019) · Duo Research Archive (University of Oslo) · Cited by 2
Difficulty Estimation for Image-Specific Medical Image Segmentation Quality Control
Fournel, J., Bartoli, A., Marchi, B., Maurin, A., Bigdeli, S., Jacquier, A., & Feragen, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05169-1_12 · Cited by 1
Lin, Y., Li, D., Chen, X., He, W., Guan, Q., Chen, D., Zhang, A., Yan, X., & Sun, Y. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.113086 · Cited by 2
Gallean, B., Laurent, T., Biasuz, K., Clement, A., Faraj, N., Lemaire, P., & Faure, E. (2025) · eLife · doi:10.7554/elife.106227.3 · Cited by 2
Poel, R., Kamath, A. J., Ermiş, E., et al. (2025) · Physics and Imaging in Radiation Oncology · doi:10.1016/j.phro.2025.100888 · Cited by 1
Han, J. S., Son, S., Park, I.-H., et al. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15135319 · Cited by 1
Quality Monitoring of Federated Covid-19 Lesion Segmentation
González, C., Harder, C. L., Ranem, A., Fischbach, R., Kaltenborn, I., Dadras, A. A., Bucher, A. M., & Mukhopadhyay, A. (2022) · Informatik aktuell · doi:10.1007/978-3-658-36932-3_8 · Cited by 1
Expert Annotation Tools for Labeling Student Capstone Project based on ACM CCS Ontology
Andre, A., Suciati, N., & Fabroyir, H. (2022) · Electrical Power, Electronics, Communications, Controls and Informatics Seminar (EECCIS) · doi:10.1109/eeccis54468.2022.9902931 · Cited by 1
Iqbal, S., Qureshi, A. N., Alhussein, M. A., Aurangzeb, K., & Wang, H. H. (2024) · Research Square · doi:10.21203/rs.3.rs-3955910/v1 · Cited by 1
Gallean, B., Laurent, T., Biasuz, K., Clement, A., Faraj, N., Lemaire, P., & Faure, E. (2025) · eLife · doi:10.7554/elife.106227.1 · Cited by 1
Gallean, B., Laurent, T., Biasuz, K., Clement, A., Faraj, N., Lemaire, P., & Faure, E. (2025) · eLife · doi:10.7554/elife.106227.2 · Cited by 1
Yazdan-Panah, A., Stankoff, B., & Colliot, O. (2024) · Medical Imaging 2024: Image Processing · doi:10.1117/12.3006580 · Cited by 1
ConfIC-RCA: Statistically Grounded Efficient Estimation of Segmentation Quality
Cosarinsky, M., Billot, R., Mansilla, L., Jimenez, G., Gaggión, N., Fu, G., Tirer, T., & Ferrante, E. (2025) · arXiv
“Reverse Classification Accuracy (RCA) [5] is a framework originally designed to predict the performance of an image segmentation method in scenarios where GT masks are unavailable.”
InfluentialStructure-guided registration in learning based image analysis
Lee, M. C. H. (2019) · Spiral (Imperial College London) · doi:10.25560/79296
Learning to Predict Error for MRI ReconstructionPreprint
Shi, H., Pezzotti, N., & Welling, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.05582
Hardware-aware Real-time Myocardial Segmentation Quality Control in Contrast EchocardiographyPreprint
Zeng, D., Ding, Y., Yuan, H., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.06909
Performance monitoring of deep learning vision systems during deployment
Rahman, Q. M. (2022) · Queensland University of Technology · doi:10.5204/thesis.eprints.229733
Liver Segmentation Quality Control in Multi-sequence MR Studies
Wang, Y., & Palma, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-18814-5_6
Multi-Organ Failure Prediction using Machine Learning Approach
Kumar, G. S. M., Impanashree, V. P., Manjunath, N., et al. (2023) · International Journal of Scientific Research in Science and Technology · doi:10.32628/ijsrst5231032
Medical volume segmentation by overfitting sparsely annotated data
Payer, T., Nizamani, F. A., Beer, M., GOETZ, M. A., & Ropinski, T. (2023) · Journal of medical imaging · doi:10.1117/1.jmi.10.4.044007 · Cited by 1
Holistic Consistency for Subject-Level Segmentation Quality Assessment in Medical Image Segmentation
Zhang, Y., Zhou, T., Chen, Q., Dou, Q., & Wang, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73158-7_9
Rockall, A., Li, X., Johnson, N. A., Lavdas, I., Santhakumaran, S., Prevost, A. T., Koh, D., Punwani, S., Goh, V., Bharwani, N., et al. (2024) · Efficacy and Mechanism Evaluation · doi:10.3310/kpwq4208 · Cited by 1
Self-citationFrom out-of-distribution detection to quality control
Lambert, B., Forbes, F., & Michel Dojat, D. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-323761-4.00014-6 · Cited by 1
Gallean, B., Laurent, T., Biasuz, K., Clement, A., Faraj, N., Lemaire, P., & Faure, E. (2025) · eLife · doi:10.7554/elife.106227
Fully automated IVUS image segmentation with efficient deep-learning-assisted annotation
Zhang, L., Chen, Z., Zhang, H., Zaman, F. A., Wahle, A., Wu, X., McNamara, C. A., Taylor, A. M., & Milan Sonka, A. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.111312
Hadler, T., Grassow, L., Kuhnt, J., Hickstein, R., Saad, H., Fenski, M., Gröschel, J., Trauzeddel, R. F., Błaszczyk, E., Ammann, C., et al. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03271-6
Şener, B. (2025) · Medical Technologies Congress (TIPTEKNO) · doi:10.1109/tiptekno68206.2025.11270118
Scale it up: Large-scale MR data processing
Langner, T. (2025) · Advances in magnetic resonance technology and applications · doi:10.1016/b978-0-443-14109-6.00017-1
Cut it here: Image segmentation for MRI
Chen, C. (., & Li, Z. (2025) · Advances in magnetic resonance technology and applications · doi:10.1016/b978-0-443-14109-6.00012-2
Senbi, A., Huang, T., Lyu, F., Li, Q., Tao, Y., Shao, W., Chen, Q., Wang, C., Wang, S., Zhou, T., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-5634-2_17
ML-Based Medical diagnosis for skin disease
Gokul, B., Venkatesan, R., Swetha Shekarappa, G., & Senbagavalli, M. (2025) · International Conference on Recent Trends in Electrical, Electronics and Computing Technologies (ICRTEECT) · doi:10.1109/icrteect67512.2025.11448705
Arega, T. W. (2023) · theses.fr (ABES) · doi:10.70675/6f691b31z766dz4a4azb504za7b14d4cf213
Robust Learning for Medical Image Segmentation
Galati, F. (2024) · theses.fr (ABES) · doi:10.70675/49df4b56zc3d3z4169z8388zbcb5428e83ca
Ribeiro, M. A. O., Gutierrez, M. A., & Nunes, F. L. S. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-01940-0
Raquel, B. F., Ribeiro, M. A. O., & Nunes, F. L. S. (2026) · Anais do Simpósio Brasileiro de Sistemas de Informação (SBSI) · doi:10.5753/sbsi.2026.248301
A Source-Free Segmentation Quality Estimation of a Model Adapted to a New Domain
Nahum, I., Nizhar, R., & Goldberger, J. (2026) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi61048.2026.11515765
Rama, E., Yu, S., Schraven, S., et al. (2026) · European Radiology Experimental · doi:10.1186/s41747-026-00742-x
Kinalioğlu, İ. H. (2026) · Pattern Recognition Letters · doi:10.1016/j.patrec.2026.08.026
Shu, S. Z., de Mortanges, A. P., Poellinger, A., Mahapatra, D., & Reyes, M. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-02257-8
Tsai, C. M., & Lee, J.-D. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.111588
Cross-Model Agreement as a Deployment-Time Reliability Signal for Automatic Polyp Segmentation
Gupta, S., & Singla, J. (2026) · arXiv · doi:10.48550/arxiv.2609.10495
“Unlike RCA [11], which requires fitting a new reverse classifier for every test image against a reference atlas, RBQE requires no per-image retraining and no atlas dependency; unlike deep ensembles, which derive disagreement from same-architecture models differing only in initialization, RBQE’s…”
Influential
Metadata
- Authors
- Valindria, V. V., Lavdas, I., Bai, W., Kamnitsas, K., Aboagye, E. O., Rockall, A. G., Rueckert, D., & Glocker, B.
- Affiliations
- Imperial College London (GB)
- Hammersmith Hospital (GB)
- Royal Marsden NHS Foundation Trust (GB)
- ORCID
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2017-08
- Volume
- 36
- Issue
- 8
- Pages
- 1597-1606
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W2594608309
- Semantic Scholar
- d11e2caf177686536879226b14e089c9382dbb30
- Cited by
- 152
- References
- 41
- Citations by year
- 2026: 12 · 2025: 21 · 2024: 14 · 2023: 21 · 2022: 20 · 2021: 15 · 2020: 15 · 2019: 21 · 2018: 12 · 2017: 1
- FWCI
- 5.45
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Spiral (Imperial College London) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Institute for Health Research under EME Project (13/122/01)
- EPSRC First Grant Scheme (EP/N023668/1)
- Indonesia Endowment for Education (LPDP)–Indonesian Presidential Ph.D. Scholarship Programme
- Imperial College President’s Ph.D. Scholarship Programme
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Engineering › Biomedical Engineering › Medical Imaging and Analysis
- Keywords
- automated image segmentation, multi-organ segmentation, overfitting, cross-validation
- MeSH terms
- Algorithms, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-22, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-22 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
