CoChat
Sign Up
Cited by 1,095Medical Image Segmentation TechniquesAdvanced Neural Network ApplicationsDomain Adaptation and Few-Shot Learning

Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation

Open access
Tajbakhsh, Nima, Jeyaseelan, Laura, Li, Qian, Chiang, Jeffrey N., Wu, Zhihao, Ding, Xiaowei·2020·Medical Image Analysis
DOI 10.1016/j.media.2020.101693
Open this paper in CoChat — free accountOpen PDF

Abstract

The medical imaging literature has witnessed remarkable progress in high-performing segmentation models based on convolutional neural networks. Despite the new performance highs, the recent advanced segmentation models still require large, representative, and high quality annotated datasets. However, rarely do we have a perfect training dataset, particularly in the field of medical imaging, where data and annotations are both expensive to acquire. Recently, a large body of research has studied the problem of medical image segmentation with imperfect datasets, tackling two major dataset limitations: scarce annotations where only limited annotated data is available for training, and weak annotations where the training data has only sparse annotations, noisy annotations, or image-level annotations. In this article, we provide a detailed review of the solutions above, summarizing both the technical novelties and empirical results. We further compare the benefits and requirements of the surveyed methodologies and provide our recommended solutions. We hope this survey article increases the community awareness of the techniques that are available to handle imperfect medical image segmentation datasets.

AI snapshot

Written from the paper’s full text.

TL;DR

This review surveys deep learning solutions for medical image segmentation when datasets have scarce annotations (limited dense labels) or weak annotations (sparse, noisy, or image-level labels). It categorizes methods, compares their data requirements and reported performance, and recommends strategies based on a cost-gain trade-off.

Key findings

  • The paper organizes solutions into two main problems: scarce annotations (Sections 4.1–4.6) and weak annotations (Sections 5.1–5.3), with subcategories including data augmentation, external labeled datasets, cost-effective annotation, unlabeled data, regularized training, post-segmentation refinement, image-level labels, sparse labels, and noisy labels.
  • For scarce annotations, the authors recommend traditional data augmentation, shape regularization, mixing augmentation, same-domain data synthesis, and CRF-based post-processing for 2D applications; self-supervised pre-training is called “one of the most promising approaches” because it requires only unlabeled data and minor architectural changes.
  • For weak annotations, the authors recommend modified CAM-based approaches for image-level labels, noting they “only fall a couple of Dice points short of using full supervision” (Feng et al., 2017); sparse-annotation methods achieve the closest performance to strongly supervised counterparts but can be iterative and less viable for small lesions.
  • Reported examples include Sakinis et al. (2019) achieving Dice 0.64 for unseen colon cancer segmentation with 1 user click versus 0.56 for the best automated model, and Kuo et al. (2018) reaching full-dataset performance with 50% of training data and 20% of annotation cost for intracranial hemorrhage segmentation.

Methods

  • Design: narrative review of deep learning methods for medical image segmentation with imperfect datasets, summarizing technical novelties and empirical results.
  • Data: published literature; no primary dataset is collected or analyzed.
  • Analysis: qualitative synthesis and comparison of surveyed methodologies by performance gain, implementation difficulty, and required data resources, with summary tables and color-coded recommendations based on cost-gain trade-off.

Limitations

  • The paper states limitations of surveyed methods, including that traditional augmentation produces correlated examples, transfer learning from natural images mainly benefits 2D models, CRF-based refinement shows mixed results in 3D, pseudo-label semi-supervised learning can be degraded by noisy model-generated annotations, and adversarial domain adaptation is “tricky to train.”
  • The paper also notes that active learning methods often neglect annotation cost, and that interactive segmentation’s efficacy in reducing annotation cost “should be corroborated through more systematic user studies.”
  • Evident from the text: the review does not describe a systematic search protocol or quantitative meta-analysis, and its recommended solutions are based on the authors’ qualitative cost-gain assessment rather than formal comparative trials.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.media.2020.101693

Cites

169 of 169 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 189 cited sources checked · 0 supported · 0 flagged

  • Findingcitation
    § 4.4.2. Semi-supervised learning with pseudo annotations · p. 16

    Nie et al. (2018) show the effectiveness of their framework over a pure supervised approach across multiple datasets.

    “The authors show the effectiveness of the suggested framework over a pure supervised approach across multiple datasets.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Nie et al. (2018)

      Not checked in the free run.

  • Findingcitation
    § 4.4.1. Self-supervised Pre-training · p. 13

    Ross et al. (2018) report that weights pre-trained via self-supervised learning tend to lose their edge over randomly initialized weights when the size of the training set changes from 25 to 400 images.

    “However, weights pre-trained via self-supervised learning tend to lose their edge over randomly initialized weights when the size of the training set changes from 25 to 400 images.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ross et al. (2018)

      Not checked in the free run.

  • Findingcitation
    § 4.4.2. Semi-supervised learning with pseudo annotations · p. 15

    Zhou et al. (2018a) report that the performance gain drops to 1 point in Dice when the size of the labeled training set is increased to 100 CT scans.

    “The performance gain however drops to 1 point in Dice when the authors increase the size of labeled training set to 100 CT scans.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhou et al. (2018a)

      Not checked in the free run.

  • Findingcitation
    § 4.4.2. Semi-supervised learning with pseudo annotations · p. 15

    Zhou et al. (2018a) report that under a data split of 50 labeled CT and 100 unlabeled CT scans, their method achieves on average four points increase in Dice over the model trained using only the labeled data.

    “The training set consists of 50 labeled CT and 100 unlabeled CT scans. Under this data split, the suggested method achieves on average four points increase in Dice over the model trained using only the labeled data.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhou et al. (2018a)

      Not checked in the free run.

  • Findingcitation
    § 4.4.2. Semi-supervised learning with pseudo annotations · p. 15

    Zhao et al. (2019b) report that on a test set with 20 MR scans, their model trained with 12 labeled and 749 unlabeled MR images achieves a 1.6 points increase in Dice over the base model trained using only labeled data.

    “The suggested model is trained for the task of brain region segmentation using 12 labeled and 749 unlabeled MR images. On a tests set with 20 MR scans, the suggested model achieves 1.6 points increase in Dice over the base model trained using only labeled data.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhao et al. (2019b)

      Not checked in the free run.

  • Classified as probably not a substantial claim. 73
  • May combine two claims. 96
  • The quote only partly states this claim. 2

  • Classified as probably not a substantial claim.

    Findingnone given
    § Abstract · p. 1

    A large body of research has studied medical image segmentation with imperfect datasets, tackling two major dataset limitations: scarce annotations and weak annotations.

    “Recently, a large body of research has studied the problem of medical image segmentation with imperfect datasets, tackling two major dataset limitations: scarce annotations where only limited annotated data is available for training, and weak annotations where the training data has only sparse annotations, noisy annotations, or image-level annotations.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given
    § 1. Introduction · p. 1

    The common limitations of medical image segmentation datasets include scarce annotations and weak annotations.

    “The common limitations of medical image segmentation datasets include scarce annotations where only limited annotated data is available for training, and weak annotations where the training data has only sparse annotations, noisy annotations, or image-level annotations.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Findingreasoning
    § 1. Introduction · p. 1

    Researchers from the medical imaging community have actively sought solutions, resulting in a diverse and effective set of techniques with demonstrated capabilities in handling scarce and weak annotations for medical image segmentation.

    “In response to this challenge, researchers from the medical imaging community have actively sought solutions, resulting in a diverse and effective set of techniques with demonstrated capabilities in handling scarce and weak annotations for the task of medical image segmentation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given
    § 4.2.2. Domain Adaptation · p. 6

    A recurring theme in many of the domain adaptation papers discussed is the use of GANs, CycleGANs, or some sort of adversarial loss for the purpose of image reconstruction.

    “A recurring theme in many of the domain adaptation papers discussed in this section is the use of GANs, CycleGANs, or some sort of adversarial loss for the purpose of image reconstruction.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation
    § 4.4.1. Self-supervised Pre-training · p. 13

    Bai et al. (2019) report that in a low data regime where the training set has fewer than 50 MR images, self-supervised pre-training achieves a significant gain over random initialization, but the performance gain becomes only marginal when the segmentation model is trained with 100 MR images.

    “In a low data regime where the training set has fewer than 50 MR images, self-supervised pre-training achieves a significant gain over random initialization. However, the performance gain becomes only marginal when the segmentation model is trained with 100 MR images.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Bai et al. (2019)

      Not checked in the free run.

  • May combine two claims.

    Findingcitation
    § 4.4.1. Self-supervised Pre-training · p. 13

    Tajbakhsh et al. (2019a) show that image orientation prediction as a surrogate task is highly effective for diabetic retinopathy classification in fundus images and lung lobe segmentation in chest CT scans.

    “The authors show that this surrogate task is highly effective for diabetic retinopathy classification in fundus images and lung lobe segmentation in chest CT scans.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Tajbakhsh et al. (2019a)

      Not checked in the free run.

  • May combine two claims.

    Findingcitation
    § 4.4.1. Self-supervised Pre-training · p. 13

    Ross et al. (2018) report that when only 25 images are used for fine-tuning, the instrument segmentation model pre-trained via self-supervised learning achieves a Dice score of 0.61, outperforming the counterpart model pre-trained using Microsoft COCO dataset and the model trained from scratch, both with a Dice score of 0.57.

    “When only 25 images are used for fine-tuning, the instrument segmentation model pre-trained via self-supervised learning achieves a Dice score of 0.61, which outperforms the counterpart model pre-trained using Microsoft COCO dataset and the model trained from scratch, both with a Dice score of 0.57.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Ross et al. (2018)

      Not checked in the free run.

  • May combine two claims.

    Findingcitation
    § 4.4.1. Self-supervised Pre-training · p. 13

    Spitzer et al. (2018) report that a segmentation model trained from a self-supervised model achieves a Dice score of 0.8, outperforming a model trained from scratch with a Dice of 0.72 for cytoarchitectonic segmentation of the human brain.

    “The segmentation model trained from the self-supervised model achieves a Dice score of 0.8, outperforming the model trained from scratch with a Dice of 0.72.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Spitzer et al. (2018)

      Not checked in the free run.

  • May combine two claims.

    Findingcitation
    § 4.4.1. Self-supervised Pre-training · p. 14

    Zhou et al. (2019b) evaluated Models Genesis for both image classification and segmentation in seven 2D and 3D medical datasets, demonstrating three and five points increase in IoU over 3D models trained from scratch for lung nodule segmentation and liver segmentation in CT scans, respectively.

    “The authors have evaluated Models Genesis for both image classification and segmentation in seven 2D and 3D medical datasets, demonstrating three and five points increase in IoU over 3D models trained from scratch for lung nodule segmentation and liver segmentation in CT scans, respectively.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhou et al. (2019b)

      Not checked in the free run.

  • May combine two claims.

    Findingcitation
    § 4.4.2. Semi-supervised learning with pseudo annotations · p. 15

    Bai et al. (2017) report that their approach is simple to implement but, hindered by the quality of pseudo annotations, the resulting model achieves only a moderate level of improvement over the model trained using only labeled data.

    “This approach is simple to implement; however, hindered by the quality of pseudo annotations, the resulting model achieves only a moderate level of improvement over the model trained using only labeled data.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Bai et al. (2017)

      Not checked in the free run.

  • May combine two claims.

    Findingcitation
    § 4.4.2. Semi-supervised learning with pseudo annotations · p. 15

    Min et al. (2018) report that their semi-supervised learning framework based on pseudo-labeling and a two-stream network is noise-resilient, significantly outperforming fully supervised models under varying levels of label noise present in labeled images.

    “The results show that the suggested semi-supervised learning framework based on pseudo-labeling and the two-stream network is noise-resilient, significantly outperforming fully supervised models under varying levels of label noise present in labeled images.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Min et al. (2018)

      Not checked in the free run.

  • May combine two claims.

    Findingcitation
    § 4.4.3. Semi-supervised learning without pseudo annotations · p. 16

    Chartsias et al. (2018) show that their framework is highly effective for myocardial segmentation in low-data regime, but the performance gap closes as the size of training set increases.

    “The authors show that the suggested framework is highly effective for myocardial segmentation in low-data regime, but the performance gap closes as the size of training set increases.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Chartsias et al. (2018)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation
    § 4.4.1. Self-supervised Pre-training · p. 13

    Tajbakhsh et al. (2019a) report that image colorization proved more effective than random initialization for skin segmentation in tele-medicine images with both small (~140 images) and large (~1400 images) training sets, but was outperformed by transfer learning from ImageNet.

    “While image colorization proved more effective than random initialization in training with both small ( ≈ 140 images) and large ( ≈ 1400 images) training sets, it was outperformed by transfer learning from ImageNet, presumably because the distance between tele-medicine skin images and ImageNet is small.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Tajbakhsh et al. (2019a)

      Not checked in the free run.

    • Rests on an assumption: The distance between tele-medicine skin images and ImageNet is small.

  • May combine two claims.

    Findingown data
    § 4.4.3. Semi-supervised learning without pseudo annotations · p. 17

    Li et al. (2019b) and Bortsova et al. (2019) methods are effective in boosting segmentation performance in low data regime, but their performance gains tend to be limited in the presence of large labeled training sets.

    “Performance evaluation on multiple datasets show that both methods above are effective in boosting segmentation performance in low data regime; however, the performance gains tend to be limited in the presence of large labeled training sets.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Li et al. (2019b)

      Not checked in the free run.

    • cites Bortsova et al. (2019)

      Not checked in the free run.

  • May combine two claims.

    Findingcitation
    § 4.4.3. Semi-supervised learning without pseudo annotations · p. 18

    Zhang et al. (2017b) report that their suggested model yields mixed results, yielding comparable Dice to a supervised model on the larger test set (60 images) while achieving 1 point increase in Dice on the smaller test set (20 images).

    “The suggested model however yields mixed results, yielding comparable Dice to a supervised model on the larger test set (60 images) while achieving 1 point increase in Dice on the smaller test set (20 images).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhang et al. (2017b)

      Not checked in the free run.

  • May combine two claims.

    Findingcitation
    § 4.4.3. Semi-supervised learning without pseudo annotations · p. 17

    Cui et al. (2019) report that their semi-supervised model achieves three points increase in Dice over a supervised model, further outperforming semi-supervised methods by Zhang et al. (2017b).

    “Test results on MR images from 30 subjects demonstrate that the semi-supervised model achieves three points increase in Dice over a supervised model, further outperforming semi-supervised methods by Zhang et al. (2017b).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Cui et al. (2019)

      Not checked in the free run.

    • cites Zhang et al. (2017b)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.1.1. Traditional Augmentation · p. 3

    Data augmentation by adjusting image quality enables the largest performance gain in MR images, with the largest improvement coming from image sharpening through the application of unsharp masking.

    “Zhang et al. (2019a) show that data augmentation by adjusting image quality enables the largest performance gain in MR images, with largest improvement coming from image sharpening through the application of unsharp masking.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhang et al. (2019a)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.1.1. Traditional Augmentation · p. 3

    Data augmentation methods used in medical imaging can be grouped by the image property they intend to manipulate.

    “The data augmentation methods used in medical imaging can be grouped by the image property they intend to manipulate Zhang et al. (2019a).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhang et al. (2019a)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 4. Problem I: Scarce annotation · p. 3

    Traditional data augmentation is handicapped by the large correlation between the original training set and the augmented examples.

    “For example, traditional data augmentation is handicapped by the large correlation between the original training set and the augmented examples.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 4. Problem I: Scarce annotation · p. 3

    Transfer learning from natural images only benefits 2D medical image segmentation models, with no benefits to the common 3D medical image segmentation models.

    “Transfer learning from natural images only benefits 2D medical image segmentation models, with no benefits to the common 3D medical image segmentation models.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § A Review of Deep Learning Solutions for Medical Image Segmentation · p. 2

    Image segmentation requires the strongest supervision among other vision tasks such as classification and detection, and is thus most vulnerable to the quality and quantity of annotations.

    “Our focus is motivated by the fact that image segmentation requires the strongest supervision among other vision tasks such as classification and detection; and thus, most vulnerable to the quality and quantity of annotations.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisereasoning
    § 4. Problem I: Scarce annotation · p. 3

    Proactive approaches actively enlarge the training set through cost-effective annotation and synthetic data generation or change the training paradigm by leveraging unlabeled data and using additional model regularization during training.

    “The proactive approaches, on the other hand, actively enlarge the training set through cost-effective annotation and synthetic data generation or change the training paradigm by leveraging unlabeled data and using additional model regularization during training.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.1.1. Traditional Augmentation · p. 3

    Common image properties for data augmentation consist of image quality, image appearance, and image layout.

    “These common image properties consists of image quality, image appearance, and image layout.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 4.1.2. Mixing Augmentation · p. 3

    Manifold mixup has recently proved effective for prostate cancer segmentation on MR image, improving Dice by two to four points depending on the neural architecture used for segmentation.

    “Manifold mixup has recently proved effective for prostate cancer segmentation on MR image Jung et al. (2019), improving Dice by two to four points depending on the neural architecture used for segmentation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Jung et al. (2019)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning
    § 1. Introduction · p. 1

    In the presence of dataset shortcomings, even the most advanced segmentation models may fail to generalize to datasets from real-world clinical settings.

    “In the presence of these dataset shortcomings, even the most advanced segmentation models may fail to generalize to datasets from real-world clinical settings.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Recent advanced segmentation models still require large, representative, and high quality annotated datasets.

  • May combine two claims.

    Premisecitation
    § 4.1.1. Traditional Augmentation · p. 3

    Traditional data augmentation has proved effective in reducing over-fitting and improving test performance for both natural and medical images.

    “Traditional data augmentation has proved effective in reducing over-fitting and improving test performance for both natural and medical images Zhang et al. (2016).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhang et al. (2016)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.2.2. Domain Adaptation · p. 8

    Zhang et al. (2018d) demonstrate that offline data augmentation may only be partially effective and in some cases can even deteriorate the performance.

    “However, Zhang et al. (2018d) demonstrate that the above offline data augmentation may only be partially effective and in some cases can even deteriorate the performance.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Zhang et al. (2018d)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.1.3. Synthetic Augmentation · p. 4

    The suggested data augmentation method enables four points increase in Dice over a model trained using traditional data augmentation and 3 points increase in Dice over atlas-based data augmentation.

    “For the task of brain structure segmentation, the suggested data augmentation method enables four points increase in Dice over a model trained using traditional data augmentation and 3 points increase in Dice over atlas-based data augmentation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhao et al. (2019a)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.2.2. Domain Adaptation · p. 8

    Dou et al. (2018) achieve substantial performance boost over single domain training and 2% improvement in Dice over other domain adaptation techniques.

    “Using 16 subjects from each modality for training and 4 for testing, they achieve substantial performance boost over single domain training and 2% improvement in Dice over other domain adaptation techniques.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dou et al. (2018)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.2.2. Domain Adaptation · p. 8

    Valindria et al. (2018) results showed that the baseline was at par with or in some cases outperformed most variants, the only exception being the fourth variant, which consistently outperformed the baseline and other dual stream variants.

    “Their results showed that the baseline was actually at par with or in some cases outperformed most variants, the only exception being the fourth variant, which consistently outperformed the baseline and other dual stream variants.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Valindria et al. (2018) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 4.2.1. Transfer Learning · p. 5

    The scarcity of transfer learning from natural images to medical image segmentation is in part due to the 3D nature of medical images, which hampers transfer learning from 2D models trained on natural images, and also partially due to the promising performance of shallower segmentation networks in medical imaging, which unlike deep models may not benefit from fine-tuning.

    “This trend is in part due to the 3D nature of medical images, which hampers transfer learning from 2D models trained on natural images, and also partially due to the promising performance of shallower segmentation networks in medical imaging, which unlike deep models may not benefit from fine-tuning.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationcitation
    § 4.2.2. Domain Adaptation · p. 7

    The intuition behind the joint optimization of image synthesis and segmentation is that the training process can benefit from the complementary information between the synthesis and segmentation networks.

    “The intuition behind this joint optimization is that the training process can benefit from the complementary information between the synthesis and segmentation networks.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning
    § 4.2.2. Domain Adaptation · p. 6

    As individual datasets tend to be small and typically originate from a single institution, they are inherently biased and the resulting models tend to perform poorly in the real world.

    “As individual datasets tend to be small and typically originate from a single institution, they are inherently biased and the resulting models tend to perform poorly in the real world.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: A frequently encountered obstacle in medical imaging is distribution shift between the data available for training and the data encountered in clinical practice.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisereasoning
    § 4.2.3. Dataset Fusion · p. 9

    Dataset fusion techniques leverage multiple datasets to train a universal segmentation model based on heterogeneous, disjoint datasets, offering two advantages: more efficient training and enhanced regularization.

    “Dataset fusion techniques leverage multiple datasets to train a universal segmentation model based on heterogeneous, disjoint datasets, offering two advantages: 1) more efficient training, as multiple models are consolidated into a single model, and 2) enhanced regularization, as data from multiple sources can provide further supervision.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.2.2. Domain Adaptation · p. 8

    Giger (2018) propose converting the CT (target) domain to the MR (source) domain and then using an existing atlas-based algorithm (MALP-EM) to perform segmentation on the converted MR images.

    “Giger (2018) propose converting the CT (target) domain to the MR (source) domain and then using an existing atlas-based algorithm (MALP-EM) to perform the segmentation on the converted MR images.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Giger (2018) source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.2.2. Domain Adaptation · p. 8

    Zhang et al. (2018d) improve the Dice score on CT images by eight points and MR images by two points over other methods that use both real and synthetic data for training.

    “They improve the Dice score on CT images by eight points and MR images by two points over other methods that use both real and synthetic data for training.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Zhang et al. (2018d)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.2.2. Domain Adaptation · p. 8

    Zhang et al. (2018c) convert annotated CT scans to digitally reconstructed radiographs (DRRs) via a 3D to 2D projection and learn a mapping between 815 DRRs and 73 training X-ray images to segment X-ray images without X-ray segmentation annotations.

    “For this purpose, the authors first convert annotated CT scans to digitally reconstructed radiographs (DRRs) via a 3D to 2D projection, and then learn a mapping between 815 DRRs and 73 training X-ray images.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhang et al. (2018c)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 4.2.2. Domain Adaptation · p. 6

    While CycleGAN provides a one-is-to-one mapping between the two domains, MUNIT allows a one-is-to-many mapping by sampling from the style encoding distribution.

    “While CycleGAN provides a one-is-to-one mapping between the two domains, MUNIT allows a one-is-to-many mapping by sampling from the style encoding distribution.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Zhu et al. (2017)

      Not checked in the free run.

    • cites Huang et al. (2018a)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.2.2. Domain Adaptation · p. 8

    Chen et al. (2018) use the CycleGAN with an additional semantic adversarial loss to distinguish between source segmentation masks and segmentation predictions of the converted target to source images.

    “Chen et al. (2018) use the CycleGAN with an additional semantic adversarial loss, which is used to distinguish between source segmentation masks and segmentation predictions of the converted target to source images.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Chen et al. (2018)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.2.2. Domain Adaptation · p. 7

    Huo et al. (2018b) report 2% improvement in Dice over the existing state of the art-a 2-stage CycleGAN followed by a separate segmentation network-for splenomegaly and total intracranial volume segmentation.

    “The authors further extend their work in Huo et al. (2018b) for the task of splenomegaly and total intracranial volume segmentation, reporting 2% improvement in Dice over the existing state of the art-a 2-stage CycleGAN followed by a separate segmentation network.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Huo et al. (2018b)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisereasoning
    § 4.2.2. Domain Adaptation · p. 8

    If segmentation masks are available for both domains, there is no longer a distinction between the choice of source and target domains.

    “If the segmentation masks are available for both domains, there is no longer a distinction between the choice of source and target domains.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.1.4. Summary · p. 5

    Image synthesis methods are more effective for handling data scarcity as well as rare conditions, but are more demanding to implement because their training schemes typically require adversarial networks and additional labeled or unlabeled datasets.

    “Although these methods are more effective for handling data scarcity as well as rare conditions, they are more demanding to implement, because their training schemes typically require adversarial networks and additional labeled or unlabeled datasets.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 4.2.1. Transfer Learning · p. 5

    Transfer learning from natural images has been widely adopted for medical image classification; however the application to medical image segmentation has been scarce.

    “Transfer learning from natural images has been widely adopted for medical image classification Tajbakhsh et al. (2016); Shin et al. (2016); however the application to medical image segmentation has been scarce.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Tajbakhsh et al. (2016) source

      Not checked in the free run.

    • cites Shin et al. (2016) source

      Not checked in the free run.

  • May combine two claims.

    Premisereasoning
    § 4.3.1. Active Learning · p. 9

    Active learning is a cost-effective approach to enlarge the training datasets and is highly amenable to the problem of limited annotation budget in medical image segmentation.

    “Active learning is a cost-effective approach to enlarge the training datasets; and thus, it is highly amenable to the problem of limited annotation budget in medical image segmentation where clinical experts have limited availability, annotation cost is high, and the amount of unlabelled data is usually non-trivial.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning
    § 4.2.4. Summary · p. 9

    The appealing feature of domain adaptation methods without target annotations is independence from the target domain annotations, which makes these methods the only viable solution to deal with unlabeled target domain images.

    “however, the appealing feature of domain adaptation methods without target annotations is independence from the target domain annotations, which makes these methods the only viable solution to deal with unlabeled target domain images.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 4.4.1. Self-supervised Pre-training · p. 13

    Self-supervised model pre-training consists of pre-training a model using unlabeled medical data, which is easier to obtain, and then fine-tuning the pre-trained model for the target medical vision task using limited labeled data.

    “The key idea consists of pre-training the model using unlabeled medical data, which is easier to obtain, and then fine-tune the pre-trained model for the target medical vision task using the limited labeled data available for training.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Unlabeled medical data is easier to obtain than labeled medical data.

  • May combine two claims.

    Premisecitation
    § 4.3.2. Interactive Segmentation · p. 12

    Wang et al. (2018a) results show that the suggested interactive segmentation method outperforms traditional interactive segmentation methods in both accuracy and speed of annotation.

    “The results show that the suggested interactive segmentation method outperforms traditional interactive segmentation methods in both accuracy and speed of annotation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wang et al. (2018a)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.3.2. Interactive Segmentation · p. 12

    Wang et al. (2018b) proposes a framework consisting of a proposal network and a refinement network where the former generates a base segmentation mask whereas the latter refines the base mask according to the suggestions provided by the user.

    “Wang et al. (2018b) proposes a framework consisting of a proposal network and a refinement network where the former generates a base segmentation mask whereas the latter refines the base mask according to the suggestions provided by the user.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wang et al. (2018b)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationown data
    § 4.4.1. Self-supervised Pre-training · p. 13

    Self-supervised learning methods can be divided into two types of surrogate tasks: image-to-scalar and image-to-image.

    “Reviewing the literature, we have identified two types of surrogate tasks: 1) image-to-scalar where an encoder network is pre-trained for a surrogate image classification or regression task; 2) image-to-image where an encoder-decoder network is pre-trained for a surrogate image regression task such as image colorization or image denoising.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.4.2. Semi-supervised learning with pseudo annotations · p. 15

    A limitation of previous semi-supervised approaches is that images with expert annotation and images with pseudo segmentation masks are treated equally during training, so errors in pseudo labels can degrade the quality of resulting models.

    “A limitation with the previous approaches is that the images that have expert annotation and images with pseudo segmentation masks are treated equally during training. As such, errors in the pseudo labels can degrade the quality of the resulting models.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 4.4.2. Semi-supervised learning with pseudo annotations · p. 14

    Semi-supervised learning with pseudo annotations has shown promising performance, producing models that outperform counterparts trained using only labeled data.

    “Semi-supervised learning with pseudo annotations has shown promising performance, producing models that outperform the counterparts trained using only labeled data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.3.2. Interactive Segmentation · p. 12

    Sun et al. (2018a) propose an interactive method for segmenting fuzzy boundaries, wherein the user first places a point roughly at the center of the object, and then the model performs object delineation for the user-specified structure.

    “Sun et al. (2018a) propose an interactive method for segmenting fuzzy boundaries, wherein the user first places a point roughly at the center of the object, and then the model performs object delineation for the user-specified structure.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sun et al. (2018a)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisereasoning
    § 4.3.1. Active Learning · p. 9

    Active learning requires the availability of a base segmentation model; thus, a minimal set of base annotations is necessary.

    “Active learning, in its general form, requires the availability of a base segmentation model; thus, a minimal set of base annotations is necessary.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 4.4.1. Self-supervised Pre-training · p. 13

    Image-to-scalar pre-training can be used to initialize the encoder of a segmentation network, with the decoder initialized with random weights.

    “While the former approach seems particularly suitable for a downstream image classification task, it can still be used to initialize the encoder of a segmentation network, in which case the decoder should be initialized with random weights.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 4.2.4. Summary · p. 9

    In general, models trained with target domain annotations are bound to generalize better to the target domain.

    “In general, the models trained with target domain annotations are bound to generalize better to the target domain”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation
    § 4.3.3. Summary · p. 13

    Active learning methods are typically iterative and slow, but recent works present a 1-shot, fast approach to active learning with comparable performance to the iterative counterparts.

    “While active learning methods are typically iterative and slow, recent works Mahapatra et al. (2018); Zheng et al. (2019) present a 1-shot, fast approach to active learning with comparable performance to the iterative counterparts, making active learning an even more attractive methodology.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Mahapatra et al. (2018)

      Not checked in the free run.

    • cites Zheng et al. (2019)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning
    § 4.3.2. Interactive Segmentation · p. 12

    Interactive segmentation can accelerate the annotation process by allowing expert annotators to interactively correct an initial segmentation mask generated by a model.

    “Interactive segmentation can accelerate the annotation process by allowing the expert annotators to interactively correct an initial segmentation mask generated by a model.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 4.2.4. Summary · p. 9

    The majority of domain adaptation techniques require some form of adversarial training, making them tricky to train.

    “The majority of domain adaptation techniques require some form of adversarial training, making them tricky to train.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning
    § Algorithm 1: Active learning · p. 11

    Sourati et al. (2018) have a limitation that computational complexity is super quadratic with respect to the number of parameters.

    “One limitation of this work, however, is that computational complexity is super quadratic with respect to the number of parameters, because the Fisher matrix has as many rows and columns as the number of parameters in the network.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sourati et al. (2018)

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data
    § 4.4. Leveraging Unlabeled Data · p. 13

    There are three scenarios in which unlabeled medical images have aided medical image segmentation: self-supervised pre-training, semi-supervised learning with pseudo labels, and semi-supervised learning without pseudo labels.

    “We have identified three scenarios wherein unlabeled medical images have aided medical image segmentation: 1) self-supervised pre-training where unlabeled images are used to pre-train a segmentation network; 2) semi-supervised learning with pseudo labels where unlabeled images are labeled by a segmentation model and then used as new examples during training; and 3) semi-supervised learning without”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given
    § 4.2.4. Summary · p. 9

    Transfer learning and dataset fusion require minimal changes to the network architecture, allowing for simple and successful joint dataset training.

    “On the other hand, transfer learning and dataset fusion require minimal changes to the network architecture, allowing for simple and successful joint dataset training.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Algorithm 1: Active learning · p. 10

    Yang et al. (2017) propose a framework called suggestive annotation where the candidate samples for each round of annotation are selected through a 2-stage screening process.

    “Yang et al. (2017) propose a framework called suggestive annotation where the candidate samples for each round of annotation are selected through a 2-stage screening process.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Yang et al. (2017)

      Not checked in the free run.

  • May combine two claims.

    Premisenone given
    § Algorithm 1: Active learning · p. 10

    Kuo et al. (2018) results are comparable to Yang et al. (2017) but are obtained using datasets that are two orders of magnitude larger in size.

    “These results are comparable to Yang et al. (2017), but are obtained using datasets that are two orders of magnitude larger in size.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Kuo et al. (2018)

      Not checked in the free run.

    • cites Yang et al. (2017)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.3.2. Interactive Segmentation · p. 12

    Sakinis et al. (2019) with only 1 user click, the semi-automated method can achieve a Dice of 0.64 for segmenting an unseen structure, colon cancer, in 126 Abdomen CT scans, outperforming the best automated model with a Dice of 0.56.

    “To put this in perspective, with only 1 user click, this semi-automated method can achieve a Dice of 0.64 for segmenting an unseen structure, colon cancer, in 126 Abdomen CT scans, outperforming the best automated model with a Dice of 0.56.”
    • Also stated in: § 4.3.3. Summary · p. 13 · plain
      “In particular, the work by Sakinis et al. (2019) was quite effective, where 1 user click led to a Dice of 0.64 for segmenting colon cancer, outperforming the best automated model with a Dice of 0.56.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Sakinis et al. (2019)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 4.3.2. Interactive Segmentation · p. 12

    Interactive segmentation complements active learning in achieving cost-effective annotation: the latter identifies which images to be annotated whereas the former reduces the time required to complete the annotation of a selected image.

    “Interactive segmentation complements active learning in achieving cost-effective annotation: the latter identifies which images to be annotated whereas the former reduces the time required to complete the annotation of a selected image.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.4.4. Summary · p. 18

    Semi-supervised schemes without pseudo annotations are effective in coping with limited labeled training sets, improving the Dice score by a couple of points in most cases.

    “Nevertheless, these semi-supervised schemes are effective in coping with limited labeled training sets, improving the Dice score by a couple of points in most cases.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 4.5.2. Multi-task Learning · p. 20

    Huang et al. (2018b) approach outperforms other U-net variants by several points and Mask R-CNN by 20 points when Dice score is used for colorectal tumor segmentation in MRI volumes.

    “This approach outperforms other U-net variants by several points and the popular Mask R-CNN algorithm by 20 points when the Dice score is used for performance evaluation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Huang et al. (2018b)

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 4.4.4. Summary · p. 18

    Semi-supervised learning without pseudo annotations typically consists of complex neural architectures because it accommodates a segmentation task for labeled data and an unsupervised task for unlabeled data.

    “Since these methods accommodate the segmentation task for the labeled data and an unsupervised task for the unlabeled data, they typically consist of complex neural architectures.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.5.1. Altered Image Representation · p. 19

    Altered image representations present deep models with an easier problem to solve, thereby reducing the need for large training sets.

    “Altered image representations consist of projecting or transforming the images into a more informative or compact representation, which present deep models with an easier problem to solve, thereby reducing the need for large training sets.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Deep supervised models with a large number of parameters are prone to over-fitting, particularly in the absence of large training sets.

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based).

    Premisecitation
    § 4.5.1. Altered Image Representation · p. 19

    Ghafoorian et al. (2017) achieved a 6% increase in Dice over their single scale baseline without handcrafted features on a test set of 46 cases.

    “On a test set of 46 cases, their multi-scale architecture equipped with the hand-crafted features achieves 6% increase in Dice over their single scale baseline that does not incorporate the handcrafted features.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Ghafoorian et al. (2017) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 4.4.3. Semi-supervised learning without pseudo annotations · p. 16

    The form of semi-supervised learning without pseudo annotations enables training the model in the source domain using both labeled and unlabeled data, increasing the effective size of the training set and mitigating the problem of limited annotations in the source domain.

    “The latter enables training the model in the source domain using both labeled and unlabeled data, increasing the effective size of the training set, mitigating the problem of limited annotations in the source domain.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation
    § 4.4.4. Summary · p. 18

    The limitation of noisy model-generated annotations has been addressed in Min et al. (2018) and Nie et al. (2018), but at the expense of relatively complex segmentation architectures.

    “This limitations has been recently addressed in Min et al. (2018); Nie et al. (2018), but at the expense of relatively complex segmentation architectures.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Min et al. (2018)

      Not checked in the free run.

    • cites Nie et al. (2018)

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation
    § 4.5.2. Multi-task Learning · p. 20

    A single model per condition is not a scalable approach for clinical use given the sheer number of potential conditions.

    “given the sheer number of potential conditions, a single model per condition is not a scalable approach for clinical use.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Harouni et al. (2018b)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.5.2. Multi-task Learning · p. 20

    Myronenko (2018) method outperforms other modifications such as CRF post-processing and test-time data augmentation, winning first place in the 2018 BraTS challenge.

    “The suggested method outperforms other modifications such as CRF post-processing as well as sophisticated test-time data augmentation, winning the first place in the 2018 BraTS challenge”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Myronenko (2018)

      Not checked in the free run.

    • cites Menze et al. (2014) source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation
    § 4.4.1. Self-supervised Pre-training · p. 14

    The Models Genesis framework is scalable to a large library of surrogate tasks because all tasks in the library can share the same encoder and decoder during training, eliminating the need for task-specific decoders.

    “The suggested framework is scalable to a large library of surrogate tasks, because all tasks in the library can share the same encoder and decoder during training, eliminating the need for task-specific decoders.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhou et al. (2019b)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 4.4.4. Summary · p. 18

    Self-supervised learning is easy to implement owing to intuitive underlying proxy tasks.

    “2) ease of implementation owing to the intuitive underlying proxy tasks”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.4.4. Summary · p. 18

    Semi-supervised methods without pseudo annotations do not attempt to generate annotations for unlabeled images and are therefore not vulnerable to annotation noise.

    “Compared to the previous strategy, these methods do not attempt to generate annotations for the unlabeled images; as such, they are not vulnerable to annotation noise.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 4.5.2. Multi-task Learning · p. 20

    Harouni et al. (2018b) single U-net model does not exceed state of the art performance for any one given modality or target, but performance is competitive across all domains.

    “While this model does not exceed the state of the art performance for any one given modality or target, performance is competitive across all domains.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Harouni et al. (2018b)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.5.2. Multi-task Learning · p. 20

    Harouni et al. (2018b) report that the presence of the image classification branch results in slightly improved segmentation performance.

    “The authors also report that the presence of the image classification branch results in slightly improved segmentation performance.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Harouni et al. (2018b)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 4.5.1. Altered Image Representation · p. 19

    Compact representations can benefit 3D applications where the curse of dimensionality requires large annotated datasets.

    “whereas compact representations can benefit 3D applications where the curse of dimensionality requires large annotated datasets.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 4.5.2. Multi-task Learning · p. 20

    Huang et al. (2018b) showed that local rather than global decoding preserves spatial details and decreases GPU footprint.

    “The authors show that the local (rather than global) decoding approach preserves spatial details as well as decreases GPU footprint.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Huang et al. (2018b)

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 4.4.3. Semi-supervised learning without pseudo annotations · p. 17

    Transformation consistency regularization can make the model behave more predictably and utilize unlabeled data available for training.

    “By imposing this additional regularization term, not only does the model behave more predictably, but also it can utilize the unlabeled data available for training.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 4.5.3. Shape Regularization · p. 21

    Kervadec et al. (2019) report an 8% gain in Dice and a 10% gain in Hausdorff score over a baseline that uses generalized Dice as the loss function on the ISLES dataset, compared to marginal improvement on the WMH dataset.

    “report an 8% gain in Dice and a 10% gain in Hausdorff score over a baseline that uses generalized Dice as the loss function on ISLES dataset compared to marginal improvement on the WMH dataset.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kervadec et al. (2019)

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 5.1.3. Summary · p. 25

    Both CAM-based and MIL-based approaches improve model performance over their respective baselines.

    “While both approaches improve the model performance over their respective baselines, it would be interesting to see how these two techniques can be combined together to achieve even higher gains.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given
    § 4.5.4. Summary · p. 22

    Multi-task learning leads to consistent improvement over single-task segmentation models and enables a consolidated framework for multi-modality multi-condition medical image segmentation.

    “We then reviewed multi-task learning for medical image segmentation, which leads to consistent improvement over the single-task segmentation models and further enables a consolidated framework for multi-modality multi-condition medical image segmentation.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 4.5.3. Shape Regularization · p. 22

    Oktay et al. (2017) attribute the inferior performance by Ravishankar et al. (2017) to over-regularization, which they overcame by replacing CDAE with a frozen encoder during training.

    “The authors attribute the inferior performance by Ravishankar et al. (2017) to over-regularization, which they have overcome by replacing CDAE with a frozen encoder during training.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Oktay et al. (2017) source

      Not checked in the free run.

    • cites Ravishankar et al. (2017)

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given
    § 4.6.4. summary · p. 24

    While both locally and fully-connected CRF have proved effective in 2D segmentation applications, their extension to 3D applications has shown only mixed results.

    “While both locallyand fully-connected CRF have proved effective in 2D segmentation applications, their extension to 3D applications has shown only mixed results.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § 4.6.2. Fully Connected CRF (FC-CRF) · p. 23

    Kamnitsas et al. (2017) is the first to extend FC-CRF to 3D brain lesion segmentation in MR Images.

    “The work by Kamnitsas et al. (2017) is the first to extend FC-CRF to 3D brain lesion segmentation in MR Images”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kamnitsas et al. (2017) source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.6.1. Locally Connected CRF · p. 23

    Roth et al. (2015) report an average of 3.3% gain in Dice using CRF, which falls short of a 6.9% gain using Gaussian smoothing, on 82 contrast-enhanced abdominal CT volumes in 4-fold cross-validation.

    “the authors evaluate on 82 contrast-enhanced abdominal CT volumes in a 4-fold cross-validation manner, and report an average of 3.3% gain in Dice using CRF that falls short of a 6.9% gain using Gaussian smoothing.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Roth et al. (2015)

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 4.6.4. summary · p. 24

    RNN-CRF operates on CNN feature maps and is trained end-to-end along with the segmentation model, thereby addressing the limitations of parameter tuning and susceptibility to image noise.

    “Finally, we reviewed RNN-CRF, which operates on CNN feature maps and is trained end-to-end along with the segmentation model, thereby addressing the limitations of parameter tuning and susceptibility to image noise.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 4.5.3. Shape Regularization · p. 21

    Zhou et al. (2019a) approach outperforms a naive partial supervision implementation by 1.69% in Dice with ResNet-101 as backbone for 2D task and by 0.45% in Dice using 3D U-Net as backbone, both based on 5-fold cross-validation.

    “Their approach outperforms a naive partial supervision implementation, which does not benefit from shape prior regularization, by 1.69% in Dice with ResNet-101 as backbone for 2D task and by 0.45% in Dice using 3D U-Net as the backbone models, both based on 5-fold cross-validation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhou et al. (2019a)

      Not checked in the free run.

    • The authors state this as an assumption: The average organ size distribution approximates what is learned from fully labeled data.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 5.1. Learning with Image Level Labels · p. 24

    Weakly supervised techniques can take advantage of bounding boxes or image level labels.

    “Weakly supervised techniques can take advantage of bounding boxes or image level labels.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given
    § 5.1.3. Summary · p. 25

    The papers reviewed in section 5.1 address image segmentation with image-level labels, the weakest form of annotation.

    “The papers reviewed in this section address the problem of image segmentation with the weakest form of annotation: image-level labels.”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Premisecitation
    § 4.5.3. Shape Regularization · p. 21

    Duan et al. (2019) report performance superior to 3D-ACNN for myocardium segmentation, achieving 4.3% increase in Dice and 4.15mm (40% less) improvement in Hausdorff distance.

    “the authors report performance superior to 3D-ACNN Oktay et al. (2017), achieving 4.3% increase in Dice and 4.15mm (40% less) improvement in Hausdorff distance .”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Duan et al. (2019) source

      Not checked in the free run.

    • cites Oktay et al. (2017) source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.5.3. Shape Regularization · p. 22

    Dalca et al. (2018) method for brain structure segmentation evaluated on 9000 MRI scans achieves Dice scores ranging between 0.50 and 0.80 without any comparison against supervised methods.

    “For the task of brain structure segmentation, the suggested method is evaluated on 9000 MRI scans, achieving Dice scores ranging between 0.50 and 0.80 without any comparison against supervised methods.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Dalca et al. (2018)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 5.1.2. Multiple Instance Learning (MIL) · p. 25

    In weakly supervised image segmentation, MIL can learn instance-level predictions, equivalently the segmentation prediction for the input image, by learning to classify the image as a whole.

    “By learning to classify the image as a whole, MIL learns to predict instancelevel predictions, or equivalently the segmentation prediction for the input image.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.6.2. Fully Connected CRF (FC-CRF) · p. 23

    Configuring 3D FC-CRF is a laborious task.

    “the authors note that configuring 3D FC-CRF is a laborious task.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kamnitsas et al. (2017) source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 4.6.4. summary · p. 24

    Locally and fully-connected CRF methods require extensive parameter tuning and may be susceptible to image noise as they operate directly on pixel information.

    “These methods further require extensive parameter tuning and may be susceptible to image noise as they operate directly on pixel information.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation
    § 4.6.3. CRF as Recurrent Neural Networks (RNN-CRF) · p. 23

    RNN-CRF organically integrates CRF with CNNs, making it possible to train the whole network in an end-to-end manner.

    “RNN-CRF organically integrates CRF with CNNs, making it possible to train the whole network in an end-to-end manner.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 4.5.3. Shape Regularization · p. 21

    Duan et al. (2019) 2.5D FCN trained on LR cases with refinement achieves comparable performance relative to the 2.5D FCN trained on HR cases, trailing by 2.0% for most regions in Dice and 0.5mm worse in Hausdorff distance.

    “the 2.5D FCN trained on the LR cases with refinement achieves comparable performance relative to the 2.5D FCN trained on the HR cases, trailing by 2.0% for most regions in Dice and 0.5mm worse in Hausdorff distance.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Duan et al. (2019) source

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given
    § 4.5.3. Shape Regularization · p. 22

    Deep shape regularization is less prone to image noise and more semantically and structurally aware than shallow approaches.

    “Compared to shallow approaches, deep regularization, a.k.a deep supervision or deep priors, is less prone to image noise and more semantically and structurally aware.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 4.6. Post segmentation refinement · p. 22

    Variants of auto-encoders have recently been adopted for automatic error correction in medical images, outperforming conditional random fields especially in the presence of domain shift and limited annotations.

    “Variants of auto-encoders have recently been adopted for automatic error correction in medical images Larrazabal et al. (2019); Tajbakhsh et al. (2019b), outperforming conditional random fields (CRFs) especially in the presence of domain shift and limited annotations.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Larrazabal et al. (2019)

      Not checked in the free run.

    • cites Tajbakhsh et al. (2019b)

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 4.5.4. Summary · p. 22

    Input space regularization and shape regularization require no further annotations and can be taken advantage of at the cost of changing the data pipeline and minor architectural changes, respectively.

    “Also, except for multi-task learning which requires additional annotations, input space regularization and shape regularization require no further annotations; and thus, can be taken advantage of at the cost of changing the data pipeline and minor architectural changes, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 4.6.2. Fully Connected CRF (FC-CRF) · p. 23

    Kamnitsas et al. (2017) 3D generalization on BraTS dataset (274 training and 110 testing) leads to marginal performance gains in Dice: 3.7% over Random Forests, 0.3% over an ensemble method, and 0.7% over their proposed architecture.

    “their 3D generalization on BraTS dataset Menze et al. (2014) (274 training and 110 testing) leads to marginal performance gains in Dice, i.e. 3.7% over Random Forests, 0.3% over an ensemble method, and merely 0.7% over their proposed architecture”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Kamnitsas et al. (2017) source

      Not checked in the free run.

    • cites Menze et al. (2014) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.6.3. CRF as Recurrent Neural Networks (RNN-CRF) · p. 23

    The improvements by 3D RNN-CRF are inconclusive.

    “The authors acknowledge that the improvements by 3D RNN-CRF are inconclusive”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Monteiro et al. (2018)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.6.2. Fully Connected CRF (FC-CRF) · p. 23

    FC-CRF has proven effective as a segmentation post-processing solution for both natural images and 2D medical images.

    “FC-CRF has proven effective as a segmentation post-processing solution for both natural images Chen et al. (2017) and 2D medical images, e.g. Fu et al. (2016b) in retinal images, Gao et al. (2016) on individual CT slices.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Chen et al. (2017) source

      Not checked in the free run.

    • cites Fu et al. (2016b)

      Not checked in the free run.

    • cites Gao et al. (2016)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.6.2. Fully Connected CRF (FC-CRF) · p. 23

    Kr ̈ ahenb ̈ uhl and Koltun (2011) algorithm reduces computational complexity from quadratic to linear in the number of pixels involved in the computation.

    “The resulting algorithm reduces the computational complexity from quadratic to linear in the number of pixels involved in the computation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kr¨ ahenb¨ uhl and Koltun (2011)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 5.1. Learning with Image Level Labels · p. 24

    The common weakly supervised approaches suggested for medical image segmentation are based on class activation maps or multiple instance learning.

    “The common weakly supervised approaches suggested for medical image segmentation are based on class activation maps or multiple instance learning.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 5.1.2. Multiple Instance Learning (MIL) · p. 25

    Jia et al.'s method achieves a 6% improvement in F-measure around boundaries and 2% overall over the baseline without area constraints and multiple-level predictions.

    “The improvement in F-measure over the baseline (without area constraints and multiple-level predictions) is 6% around the boundaries and 2% overall.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Jia et al. (2017) source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.6.1. Locally Connected CRF · p. 23

    CRF reduces the standard deviation of Dice in all experiments, demonstrating its regularization capability in reducing inference variance.

    “However, CRF does reduce the standard deviation of Dice in all experiments, demonstrating its regularization capability in reducing inference variance.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Roth et al. (2015)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 5.1.2. Multiple Instance Learning (MIL) · p. 25

    Campanella et al.'s RNN classifier achieves a 1% increase in AUC over using the MIL approach directly.

    “They achieve a 1% increase in AUC when using the RNN classifier over just using the MIL approach directly.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Campanella et al. (2019) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § 5.3. Learning with Noisy Labels · p. 27

    This type of annotation noise can be simulated by representing labels as a polygon and reducing the number of polygon vertices, creating segmentation error in peripheral areas of masks.

    “This type of annotation noise can be simulated by representing the labels as a polygon and then reducing the number of polygon vertices, which has the effect of creating segmentation error in the peripheral areas of segmentation masks.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given
    § 6. Discussion · p. 28

    Medium-data-requirement solutions require access to additional labeled or unlabeled training data from the same or a similar domain.

    “Solutions with medium data requirements: This set of methodologies requires access to additional labeled or unlabeled training data from the same or a similar domain.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 5.2.3. Summary · p. 27

    The papers covered in section 5.2 deal with incomplete pixel labels and follow one of two distinct schools of thought: those that attempt to artificially complete labels for unlabeled pixels and those that do not.

    “The papers covered in this section deal with incomplete pixel labels and follow one of two distinct schools of thought: those that attempt to artificially complete the labels for the unlabeled pixels (Section 5.2.1) and those that do not (Section 5.2.2).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 5.2.1. Selective Loss with Mask Completion · p. 26

    Can et al. (2018) use a random walk image segmentation approach with a high threshold to perform region growing around seed scribble annotations.

    “To generate the initial ground truth, Can et al. (2018) use a random walk image segmentation approach with a high threshold to perform region growing around seed scribble annotations.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Can et al. (2018)

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 5.2.3. Summary · p. 27

    Methods that attempt to complete labels tend to be iterative, making them slow to train yet fairly reliable, because only confidently labeled pixels are added to the labeled set in each iteration.

    “The former tends to be iterative, making these methods slow to train yet fairly reliable, because only confidently labeled pixels are added to the set of labeled pixels in each iteration.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 5.2.2. Selective Loss without Mask Completion · p. 27

    Bokhorst et al. show that using instance-based balancing improves Dice score by 1% and mini-batch balancing improves it by 4% when trained entirely on sparsely annotated images.

    “The authors show that using instance-based balancing improves the Dice score by 1% and mini-batch balancing improves it by 4% when trained entirely on sparsely annotated images.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Bokhorst et al. (2018)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 5.2.2. Selective Loss without Mask Completion · p. 27

    Zhu et al.'s results show that as label noise increases, the baseline segmentation model performs significantly worse, but their model with QAM-weighted segmentation loss retains high accuracy.

    “Their results show that as the level of label noise increases, the baseline segmentation model starts to perform significantly worse, but their model with QAM-weighted segmentation loss retains its high accuracy.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Zhu et al. (2019)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 5.2.1. Selective Loss with Mask Completion · p. 26

    The papers in section 5.2.1 all attempt to artificially reconstruct the incomplete regions of the ground truth masks and use the completed masks for training.

    “The papers discussed in this subsection all attempt to artificially reconstruct the incomplete regions of the ground truth masks and use the completed masks for training.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 5.2.2. Selective Loss without Mask Completion · p. 27

    Zhu et al. (2019) introduce a quality awareness module (QAM) that is a CNN trained in parallel with the segmentation network to assess mask quality.

    “Zhu et al. (2019) introduce a new quality awareness module (QAM), which is a CNN trained in parallel with the segmentation network to assess the quality of the masks.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhu et al. (2019)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § 6. Discussion · p. 28

    Of the suggested methodologies, CRF-based post-processing has shown mixed results for 3D segmentation, and altered 3D image representations have achieved medium gains at the price of training several 2D models.

    “Of the suggested methodologies, CRF-based post-processing has shown mixed results for 3D segmentation, and altered 3D image representations have achieved medium gains at the price of training several 2D models.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given
    § 5.3.3. Summary · p. 28

    Iterative noisy mask refinement approaches are slow to train but reduce adverse effects of label noise to a minimum.

    “The former approaches are iterative and slow to train, reducing the adverse effects of label noise on the segmentation model to a minimum.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationnone given
    § 6. Discussion · p. 28

    Low-data-requirement solutions rely solely on the available labeled segmentation dataset, requiring no additional labeled or unlabeled training data.

    “Solutions with low data requirements: This group of solutions rely solely on the available labeled segmentation dataset, requiring no additional labeled or unlabeled training data.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 6. Discussion · p. 28

    A common problem with medical datasets, particularly segmentation datasets, is annotation noise where annotated contours may not always follow the contours of the region of interest.

    “A common problem with medical datasets and in particular segmentation datasets is annotation noise where the annotated contours may not always follow the contours of the region of interest.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Annotation noise is a common problem with medical datasets and in particular segmentation datasets.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given
    § 6. Discussion · p. 28

    Domain adaptation techniques are effective but difficult to adopt due to instability of adversarial training, which lies at the core of these methodologies.

    “Domain adaptation techniques are effective, but they can be difficult to adopt due to the instability of adversarial training, which lies at the core of these methodologies.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Adversarial training instability lies at the core of domain adaptation methodologies.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning
    § 5.3. Learning with Noisy Labels · p. 27

    Label noise could arise in semi-supervised learning with pseudo annotations where the model learns from its own predictions on unlabeled data.

    “Label noise could also arise in semi-supervised learning with pseudo annotations (Section 4.4.2) where the model learns from its own predictions on unlabeled data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

    Interpretationreasoning
    § 6. Discussion · p. 28

    The methodologies suggested for scarce annotations can be placed in three broad categories according to data requirements: low, medium, and high.

    “As indicated by the color encoding, the methodologies suggested for the problem of scarce annotations can be placed in three broad categories according to the data requirements:”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 6. Discussion · p. 28

    Self-supervised pre-training requires only unlabeled data and typically only minor modifications to the architecture.

    “In our opinion, self-supervised pre-training is one of the most promising approaches in this category, requiring only unlabeled data and typically only minor modifications to the architecture.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Self-supervised pre-training requires only unlabeled data and typically only minor modifications to the architecture.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 6. Discussion · p. 28

    High-data-requirement solutions require access to medical experts, but use expert knowledge in a cost-effective manner.

    “Solutions with high data requirements: These solutions require access to medical experts, but their elegance lies in the use of expert knowledge in a cost-effective manner.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 6. Discussion · p. 28

    Two solutions in the high-data-requirement category are active learning and interactive segmentation, where active learning determines which samples to annotate and interactive segmentation helps experts complete annotation tasks quickly.

    “Two solutions in this category are active learning and interactive segmentation where the former determines which samples to be annotated by experts whereas the latter helps experts complete the annotation tasks quickly.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Active learning determines which samples to be annotated by experts and interactive segmentation helps experts complete annotation tasks quickly.

  • May combine two claims.

    Interpretationcitation
    § longer training periods. · p. 29

    Modified CAM-based approaches with image-level labels show large improvement in Dice over direct CAM approaches and only fall a couple of Dice points short of using full supervision with strong annotations.

    “Not only do they use the least expensive form of annotation, but they also show large improvement in Dice over the direct CAM approaches and only fall a couple of Dice points short of using full supervision with strong annotations Feng et al. (2017).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Feng et al. (2017)

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationnone given
    § 6. Discussion · p. 28

    The methodologies suggested for handling weak annotations are closely related to the types of annotations that are readily available for training.

    “The methodologies suggested for handling weak annotations are closely related to the types of annotations that are readily available for training.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 5.3.2. Robust Loss with Iterative Mask Refinement · p. 27

    Min et al. (2018) propose a two stream network with independent weights whose concord determines the quality of segmentation mask.

    “Min et al. (2018) propose a two stream network with independent weights whose concord determine the quality of segmentation mask.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Min et al. (2018)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 6. Discussion · p. 28

    Of the suggested methodologies, semi-supervised learning with pseudo annotations has shown mixed results, with exceptions being methods that adopt advanced architectures to handle annotation noise in model-generated annotations.

    “Of the suggested methodologies, semi-supervised learning with pseudo annotations has shown mixed results with the exceptions being methods that adopt advanced architectures to handle annotation noise in model-generated annotations.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation
    § 6. Discussion · p. 29

    Dot grids may be useful for larger organ segmentation but would not be as effective for segmenting small lesions.

    “For instance, while dot grids Silvestri and Antiga (2018) may be useful for larger organ segmentation, they would not be as effective for segmenting small lesions.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Silvestri and Antiga (2018)

      Not checked in the free run.

    • Rests on an assumption: Dot grids may be useful for larger organ segmentation but would not be as effective for segmenting small lesions.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § 6. Discussion · p. 28

    Multi-task learning and dataset fusions are both straightforward solutions with reasonable performance gains.

    “Multi-task learning and dataset fusions are both straightforward solutions with reasonable performance gains.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 5.3.2. Robust Loss with Iterative Mask Refinement · p. 27

    A proper handling of label noise is also studied in semi-supervised learning with pseudo annotations where model-generated annotations, commonly corrupted by noise, are used to fine-tune the model iteratively.

    “A proper handling of label noise is also studied in the context of semi-supervised learning with pseudo annotations where the model-generated annotations, commonly corrupted by noise, are used to fine-tune the model in an iterative manner.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning
    § 5.3.2. Robust Loss with Iterative Mask Refinement · p. 27

    In doing so, the model becomes capable of predicting more accurate labels for unlabeled data.

    “In doing so, the model becomes capable of predicting more accurate labels for unlabeled data.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation
    § 5.2.2. Selective Loss without Mask Completion · p. 26

    Silvestri and Antiga (2018) recommend using a hexagonal grid for sparse annotations and show that a dense mask is not a requirement for training an effective pancreas segmentation model for abdominal CT scans.

    “Silvestri and Antiga (2018) recommend using a hexagonal grid for sparse annotations and show that a dense mask is not a requirement for training an effective pancreas segmentation model for abdominal CT scans.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Silvestri and Antiga (2018)

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 5.3. Learning with Noisy Labels · p. 27

    For medical images, label noise could be induced by annotators unintentionally (random errors) or by inconsistencies between different readers due to human subjectivity concerning ambiguous lesions (expertise errors).

    “For medical images in particular, label noise could be induced by annotators unintentionally (random errors), or by inconsistencies between different readers due to human subjectivity concerning ambiguous lesions (expertise errors) Gu et al. (2018).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gu et al. (2018)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning
    § 6. Discussion · p. 28

    Semi-supervised learning without pseudo annotations requires only additional unlabeled data and is typically less demanding to implement compared to unsupervised domain adaptation methods.

    “Semi-supervised learning without pseudo annotations require only additional unlabeled data and are typically less demanding to implement compared to unsupervised domain adaptation methods.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 5.3.3. Summary · p. 28

    The papers reviewed in section 5.3 either attempt to refine noisy masks or leave them as is.

    “Similar to the methods for handling sparse labels (Section 5.2), the papers reviewed in this section either attempt to refine the noisy masks (Section 5.3.2) or leave them as is (Section 5.3.1).”
    Quote in the paperNot a substantial claim
142 of 142

  • The paper relies on this without support.

    Recent advanced segmentation models still require large, representative, and high quality annotated datasets.

    “Despite the new performance highs, the recent advanced segmentation models still require large, representative, and high quality annotated datasets.”
    “Although the architectural advancements have enabled new performance highs, they still require large, high-quality annotated datasets-more so than before.”

  • The paper relies on this without support.

    Rarely do we have a perfect training dataset, particularly in medical imaging, where data and annotations are both expensive to acquire.

    “However, rarely do we have a perfect training dataset, particularly in the field of medical imaging, where data and annotations are both expensive to acquire.”
    “However, rarely do we have a perfectly-sized and carefully-labeled dataset to train an image segmentation model, particularly for medical imaging applications, where both data and annotations are expensive to acquire.”

  • The paper relies on this without support.

    Scarce annotation is a common problem when using supervised deep learning methods for medical image segmentation.

    “Scarce annotation is a common problem when using supervised deep learning methods for medical image segmentation.”

  • The paper relies on this without support.

    A frequently encountered obstacle in medical imaging is distribution shift between the data available for training and the data encountered in clinical practice.

    “A frequently encountered obstacle in medical imaging is that of a distribution shift between the data available for training and the data encountered in clinical practice.”

  • The paper relies on this without support.

    Unlabeled medical data is easier to obtain than labeled medical data.

    “pre-training the model using unlabeled medical data, which is easier to obtain”

  • The paper relies on this without support.

    The teacher network typically outperforms the student network.

    “This is because the teacher network typically outperforms the student network; and thus, it can generate more reliable targets for the consistency loss.”

  • The paper relies on this without support.

    The distance between tele-medicine skin images and ImageNet is small.

    “presumably because the distance between tele-medicine skin images and ImageNet is small.”

  • The paper relies on this without support.

    Creating manual segmentation masks, also known as strong annotations, is time consuming and tedious, particularly for 3D images.

    “Creating manual segmentation masks, also known as strong annotations, is time consuming and tedious, particularly for 3D images.”

  • The paper relies on this without support.

    Weak annotations can be obtained at significantly lower annotation cost than strong annotations.

    “To combat the high cost associated with strong annotations, researchers have recently explored the use of weak annotations, which can be obtained at significantly lower annotation cost.”

  • The paper relies on this without support.

    Deep supervised models with a large number of parameters are prone to over-fitting, particularly in the absence of large training sets.

    “Having a large number of parameters, deep supervised models are prone to over-fitting, particularly in the absence of large training sets.”

  • The authors state this as an assumption.

    The average organ size distribution approximates what is learned from fully labeled data.

    “the proposed method assumes that the average organ size distribution approximates what is learned from fully labeled data”

  • The paper relies on this without support.

    Scribbles are a user-friendly alternative to overlapping bounding boxes.

    “Scribbles have been recognized as a user-friendly alternative to overlapping bounding boxes.”

  • The paper relies on this without support.

    Sparse annotations are easier to obtain than strong segmentation masks.

    “even though sparse annotations are easier to obtain than strong segmentation masks”

  • The paper relies on this without support.

    Image-level annotation incurs the least annotation cost among weak annotations.

    “Of the weak annotations reviewed, image-level annotation incur the least annotation cost.”

  • The paper relies on this without support.

    Adversarial training instability lies at the core of domain adaptation methodologies.

    “due to the instability of adversarial training, which lies at the core of these methodologies.”

  • The paper relies on this without support.

    Self-supervised pre-training requires only unlabeled data and typically only minor modifications to the architecture.

    “requiring only unlabeled data and typically only minor modifications to the architecture.”

  • The paper relies on this without support.

    Active learning determines which samples to be annotated by experts and interactive segmentation helps experts complete annotation tasks quickly.

    “where the former determines which samples to be annotated by experts whereas the latter helps experts complete the annotation tasks quickly.”

  • The paper relies on this without support.

    Annotation noise is a common problem with medical datasets and in particular segmentation datasets.

    “A common problem with medical datasets and in particular segmentation datasets is annotation noise”

  • The paper relies on this without support.

    Dot grids may be useful for larger organ segmentation but would not be as effective for segmenting small lesions.

    “while dot grids Silvestri and Antiga (2018) may be useful for larger organ segmentation, they would not be as effective for segmenting small lesions.”

  • The paper relies on this without support.

    The sparse annotation process is still not entirely user-friendly.

    “the annotation process is still not entirely user-friendly”
20 of 20

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Methods for leveraging unlabeled data to create better classification models with fewer annotated datasets, e.g. with self-training and active learning, have been proposed.

    “Methods for leveraging unlabeled data to create better classification models with fewer annotated datasets, e.g. with self-training (Cheplygina et al. , 2019) and active learning (Tajbakhsh et al., 2020) have been proposed.”

    From SegQC: a segmentation network-based framework for multi-metric segmentation quality control and segmentation error detection in volumetric medical images (2024) · cites this paper as Tajbakhsh et al., 2020

    Section 4.3 (p. 9) says active learning selects unlabeled medical images for annotation to maximize model performance under a limited annotation budget. Section 4.3.1 (p. 9) places this specifically in medical image segmentation, not classification. The citation supports the active-learning example, but not the claim about better classification models; self-training is attributed to a different citation. (LLM, conf=0.90)

    Experimental — accuracy not yet measured

  • Retinal layer segmentation has an associated subjectivity, with specialists often preferring different but valid solutions in severe pathological regions.

    “Also, retinal layer segmentation, similarly to other medical segmentation tasks, has an associated subjectivity, with specialists often preferring different but valid solutions in severe pathological regions.”

    From Interactive Deep Learning-Based Retinal OCT Layer Segmentation Refinement by Regressing Translation Maps (2024) · cites this paper as [16]

    Section 5.3 (p. 27) says medical-image segmentation labels can differ between readers because of human subjectivity about ambiguous lesions. It does not establish that specialists often prefer different but valid retinal-layer segmentations specifically, or that this occurs in severe pathological regions. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Obtaining a precise and personalized layer boundary still requires physicians to actively correct different regions on multiple layers.

    “Because of this, obtaining a precise and personalized layer boundary still requires physicians to actively correct different regions on multiple layers [16].”

    From Interactive Deep Learning-Based Retinal OCT Layer Segmentation Refinement by Regressing Translation Maps (2024) · cites this paper as [16]

    Section 4.3.2 (p. 12) says expert annotators can correct model-generated masks interactively and that volumetric images involve delineation across multiple slices. It does not establish that precise, personalized layer boundaries necessarily require physicians to correct different regions on multiple layers. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Most state-of-the-art automatic segmentation methods for volumetric scans are based on deep neural networks that require a large, high-quality dataset of expert-validated annotations, which is expensive and very difficult to obtain.

    “Most state-of-the-art automatic segmentation methods for volumetric scans are based on deep neural networks that require a large, high-quality dataset of expertvalidated annotations, which is expensive and very difficult to obtain (Tajbakhsh et al., 2020).”

    From SegQC: a segmentation network-based framework for multi-metric segmentation quality control and segmentation error detection in volumetric medical images (2024) · cites this paper as Tajbakhsh et al., 2020

  • The choice of data preprocessing techniques such as normalization, augmentation, or cropping can influence the model's output by emphasizing certain features over others.

    “Additionally, the choice of data preprocessing techniques, such as normalization, augmentation, or cropping, can also influence the model’s output by emphasizing certain features over others.28”

    From Bias in artificial intelligence for medical imaging: fundamentals, detection, avoidance, mitigation, challenges, ethics, and prospects (2024) · cites this paper as [28]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Summary reverses the target-label domain-adaptation categories

    Minor

    The summary says adaptation *with* target annotations uses translation in the absence of those annotations, and adaptation *without* them learns shared features. Section 4.2.2 assigns translation to the without-target-labels case and shared feature encoding to the with-target-labels case.

1 of 1

Cites

Flagged first, then by citations

111 of the 280 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Tajbakhsh, N., Jeyaseelan, L., Li, Q., Chiang, J. N., Wu, Z., & Ding, X.
Corresponding author
Tajbakhsh, Nima
ORCID
Venue
Medical Image Analysis
Publisher
Elsevier BV
Published
2020-07
Volume
63
Pages
101693
ISSN
1361-8415 (ISSN-L)
DOI
10.1016/j.media.2020.101693
OpenAlex
W2971013993
Semantic Scholar
374b4b7db9bc7cfff74dd37c8dadccbd67e16f87
Cited by
1,095
References
280
Citations by year
2026: 174 · 2025: 197 · 2024: 180 · 2023: 152 · 2022: 168 · 2021: 155 · 2020: 66 · 2019: 3
FWCI
52.38
Type
journal article
Language
en
Open access
Licence
Locations
  • Medical Image Analysis (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
medical image segmentation, weak annotation, noisy annotations, image-level annotations, semi-supervised learning, convolutional neural networks
MeSH terms
Deep Learning, Diagnostic Imaging, Humans, Neural Networks, Computer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review