CoChat
Sign Up
Cited by 507Medical Image Segmentation TechniquesAdvanced Neural Network ApplicationsBrain Tumor Detection and Classification

DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation

Open access
Wang, Guotai, Zuluaga, Maria A., Li, Wenqi, Pratt, Rosalind, Patel, Premal A., Aertsen, Michael, Doel, Tom, David, Anna L. +3 more·2019·IEEE Transactions on Pattern Analysis and Machine Intelligence
DOI 10.1109/tpami.2018.2840695PMC PMC6594450
Open this paper in CoChat — free accountOpen PDF

Abstract

Accurate medical image segmentation is essential for diagnosis, surgical planning and many other applications. Convolutional Neural Networks (CNNs) have become the state-of-the-art automatic segmentation methods. However, fully automatic results may still need to be refined to become accurate and robust enough for clinical use. We propose a deep learning-based interactive segmentation method to improve the results obtained by an automatic CNN and to reduce user interactions during refinement for higher accuracy. We use one CNN to obtain an initial automatic segmentation, on which user interactions are added to indicate mis-segmentations. Another CNN takes as input the user interactions with the initial segmentation and gives a refined result. We propose to combine user interactions with CNNs through geodesic distance transforms, and propose a resolution-preserving network that gives a better dense prediction. In addition, we integrate user interactions as hard constraints into a back-propagatable Conditional Random Field. We validated the proposed framework in the context of 2D placenta segmentation from fetal MRI and 3D brain tumor segmentation from FLAIR images. Experimental results show our method achieves a large improvement from automatic CNNs, and obtains comparable and even higher accuracy with fewer user interventions and less time compared with traditional interactive methods.

AI snapshot

Written from the paper’s full text.

TL;DR

DeepIGeoS is a two-stage interactive segmentation framework in which a CNN proposes an initial segmentation and a second CNN refines it using user clicks/scribbles encoded as geodesic distance maps, with a trainable CRF that can enforce user interactions as hard constraints. It improved over automatic CNNs and achieved comparable or higher accuracy with fewer interactions and less user time than traditional interactive methods in 2D fetal MRI placenta and 3D brain tumor segmentation.

Key findings

  • In 2D placenta segmentation, 2D P-Net + CRF-Net(f) reached Dice 85.86 ± 11.67% and ASSD 1.85 ± 1.30 pixels; after interactive refinement, 2D R-Net + CRF-Net(fu) reached Dice 89.31 ± 5.33% and ASSD 1.22 ± 0.55 pixels.
  • In 3D brain tumor segmentation, 3D P-Net + CRF-Net(f) reached Dice 87.55 ± 6.72% and ASSD 2.04 ± 1.70 pixels; after refinement, 3D R-Net + CRF-Net(fu) reached Dice 89.93 ± 6.49% and ASSD 1.43 ± 1.16 pixels.
  • Geodesic distance maps outperformed Euclidean distance maps for refinement, and CRF-Net(fu) further improved results; CRF-Net(f) with freeform pairwise potentials outperformed Dense CRF and CRF-Net(g) in the reported comparisons.
  • Compared with traditional interactive methods, DeepIGeoS required far fewer user interactions and less user time; for 3D brain tumor segmentation, user time was about one third of that for GeoS and ITK-SNAP.

Methods

  • Two-stage pipeline: P-Net produces an automatic initial segmentation; R-Net takes the original image, initial segmentation, and user interactions converted into geodesic distance maps to refine the result.
  • Networks use a resolution-preserving dilated-convolution structure adapted from VGG-16, with multi-scale feature concatenation; a back-propagatable CRF-Net uses freeform pairwise potentials and, in R-Net, user scribbles as hard constraints.
  • Data: 2D placenta from T2-weighted fetal MRI of 25 pregnant women, with 17 volumes/624 slices for training, 3 volumes/122 slices for validation, and 5 volumes/179 slices for testing; 3D brain tumor from the 2015 BraTS training set, using FLAIR images only, with 234 cases for training and 40 for testing. Evaluation used Dice and ASSD, with Student’s t-test for significance.

Limitations

  • The paper does not include a dedicated limitations section; it states that the framework can be extended to deal with multiple organs in the future.
  • Evident from the experiments: validation is limited to two binary segmentation tasks, and the 3D experiment uses only FLAIR images and segments only the whole tumor, not multi-label or multi-modality tumor subregions.
  • Evident from the text: R-Net training uses simulated user interactions generated from mis-segmented regions, which may not fully match real user behavior.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/tpami.2018.2840695

Cites

74 of 74 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 70 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    DeepIGeoS achieves a large improvement from automatic CNNs.

    “Experimental results show our method achieves a large improvement from automatic CNNs”
    Quote in the paperSubstantial claim
  • Findingown data

    3D P-Net outperforms DeepMedic, HighRes3DNet, and 3D P-Net(b5) for 3D brain tumor segmentation Stage 1.

    “3D P-Net outperforms these three counterparts”
    Quote in the paperNot a substantial claim
  • Findingown data

    3D P-Net + CRF-Net(f) significantly improves over 3D P-Net (p < 0.05) for 3D brain tumor segmentation Stage 1.

    “only CRF-Net(f) obtains significantly better segmentation than 3D P-Net”
    Quote in the paperSubstantial claim
  • Findingown data

    CRF-Net(fu) can further improve the segmentation in Stage 2 of 2D placenta segmentation.

    “CRF-Net(fu) can further improve the segmentation.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Simulated user interactions (random clicks in mis-segmented regions) are representative of real user interactions for training R-Net.
    • Rests on an assumption: The manually delineated ground truth is accurate.
  • Findingown data

    DeepIGeoS requires no initial scribbles for 2D placenta segmentation.

    “Notice no initial scribbles are needed for DeepIGeoS.”
    Quote in the paperSubstantial claim
  • May combine two claims. 31
  • The quote only partly states this claim. 6
  • Classified as probably not a substantial claim. 26
  • Borderline: one of the tests for a substantial claim is unclear. 1

  • May combine two claims.

    Findingown data

    DeepIGeoS obtains comparable and even higher accuracy with fewer user interventions and less time compared with traditional interactive methods.

    “obtains comparable and even higher accuracy with fewer user interventions and less time compared with traditional interactive methods.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    HighRes3DNet and 3D P-Net(b5) achieve similar performance, and they are better than DeepMedic for 3D brain tumor segmentation Stage 1.

    “HighRes3DNet and 3D P-Net(b5) achieve similar performance, and they are better than DeepMedic.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    DeepIGeoS requires far fewer user interactions and less user time than other interactive methods for 2D placenta segmentation.

    “DeepIGeoS requires far fewer user interactions and less user time.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingnone given

    CRF-Net(f) is better than CRF-Net(g) in Stage 1 of 2D placenta segmentation.

    “In addition, CRF-Net(f) is better than CRF-Net(g).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The manually delineated ground truth is accurate.

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    3D P-Net has far fewer parameters compared with HighRes3DNet.

    “the proposed 3D P-Net has far fewer parameters compared with HighRes3DNet.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    DeepIGeoS only needs two short strokes to get an accurate segmentation for 2D placenta segmentation, while other interactive methods require far more scribbles to get similar results.

    “DeepIGeoS only needs two short strokes to get an accurate segmentation, while the other methods require far more scribbles to get similar results.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    2D R-Net + CRF-Net(fu) significantly improves over 2D R-Net (p < 0.05) for 2D placenta segmentation.

    “Significant improvement from 2D R-Net”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    3D P-Net achieves a Dice score of 86.68 ± 7.67% for 3D brain tumor segmentation Stage 1.

    “3D P-Net outperforms these three counterparts with 86.68”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    2D R-Net with geodesic distance performs better than min-cut user-editing and 2D R-Net(Euc) that uses euclidean distance in Stage 2 of 2D placenta segmentation.

    “Fig. 8 shows 2D R-Net with geodesic distance performs better than min-cut user-editing and 2D R-Net(Euc) that uses euclidean distance.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Simulated user interactions (random clicks in mis-segmented regions) are representative of real user interactions for training R-Net.
    • Rests on an assumption: The manually delineated ground truth is accurate.

  • May combine two claims.

    Findingown data

    The proposed method achieves better results than automatic CNNs for placenta from 2D fetal MRI and brain tumors from 3D FLAIR images.

    “our proposed method achieves better results than automatic CNNs.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    3D R-Net + CRF-Net(fu) significantly improves over 3D R-Net (p < 0.05) for 3D brain tumor segmentation Stage 2.

    “Significant improvement from 3D R-Net”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    3D R-Net with geodesic distance and CRF-Net(fu) achieves an ASSD of 1.43 ± 1.16 pixels for 3D brain tumor segmentation Stage 2.

    “ASSD of 1.43”
    Quote in the paperSubstantial claim

  • Borderline on S4 (based): unclear from the text.

    Findingnone given

    The proposed method requires far less user time compared with traditional interactive methods.

    “It requires far less user time compared with traditional interactive methods”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    3D R-Net with geodesic distance and CRF-Net(fu) achieves a Dice score of 89.93 ± 6.49% for 3D brain tumor segmentation Stage 2.

    “Dice score of 89.93”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisenone given

    Euclidean distance treats each direction equally and does not take the image context into account.

    “However, the euclidean distance does not take advantage of image context information.”
    • Also stated in: § User Interaction-Based Geodesic Distance Maps · plain
      “However, the euclidean distance treats each direction equally and does not take the image context into account.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    In [40], a deep interactive object selection method transforms user-provided clicks into euclidean distance maps and concatenates them with the input of FCNs.

    “In [40], a deep interactive object selection method was proposed where user-provided clicks are transformed into euclidean distance maps and then concatenated with the input of FCNs.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [40] Xu N., Price B., Cohen S., Yang J., and Huang T., “ Deep interactive object selection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 373– 381. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given

    DeepIGeoS is efficient and can respond to user interactions in real time, leading to very short user time.

    “Third, it is efficient and can respond to user interactions in real time, which leads to very short user time.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    DeepCut employs user-provided bounding boxes as annotations to train CNNs for the segmentation of fetal MRI.

    “DeepCut [32] employs user-provided bounding boxes as annotations to train CNNs for the segmentation of fetal MRI.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [32] Rajchl M., Lee M., Oktay O., Kamnitsas K., Passerat-Palmbach J., Bai W., Rutherford M., Hajnal J., Kainz B., and Rueckert D., “ DeepCut: Object segmentation from bounding box annotations using convolutional neural networks,” IEEE Trans. Medical Imaging, vol. 36, no. 2, pp. 674– 683, Feb. 2017. 27845654 PMC7115996 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    CNNs have achieved state-of-the-art performance in many image analysis applications.

    “Recently, deep learning with convolutional neural networks (CNNs) has achieved the state-of-the-art performance in many image analysis applications [10].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [10] Litjens G., Kooi T., Bejnordi B. E., Setio A. A. A., Ciompi F., Ghafoorian M., van der Laak J. A., Ginneken B. V., and Sánchez C. I., “ A survey on deep learning in medical image analysis,” Med. Image Anal., vol. 42, pp. 60– 88, 2017. 28778026 source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    SlicSeg accepts user-provided scribbles in a single start slice to train an Online Random Forest for 3D segmentation, but lacks flexibility to allow further user-editing.

    “SlicSeg [6] accepts user-provided scribbles in a single start slice to train an Online Random Forest for 3D segmentation, but lacks in flexibility to allow further user-editing.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [6] Wang G., Zuluaga M. A., Pratt R., Aertsen M., Doel T., Klusmann M., David A. L., Deprest J., Vercauteren T., and Ourselin S., “ Slic-Seg: A minimally interactive segmentation of the placenta from sparse and motion-corrupted fetal MRI in multiple views,” Med. Image Anal., vol. 34, pp. 137– 147, 2016. 27179367 PMC5052128 source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    ITK-SNAP requires most user interactions at the beginning and its active contour model can hardly be refined with additional user interactions once an initial segmentation is obtained.

    “For example, the widely used ITK-SNAP [5] takes user-provided seed pixels or blobs as a starting point and employs an active contour model for segmentation. It requires most of the user interactions to be given at the beginning and the underpinning model can hardly be refined with additional user interactions once an initial segmentation is obtained.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [5] Yushkevich P. A., Piven J., Hazlett H. C., Smith R. G., Ho S., Gee J. C., and Gerig G., “ User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability,” NeuroImage, vol. 31, no. 3, pp. 1116– 1128, 2006. 16545965 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Typical CNNs were originally designed for image classification tasks.

    “Typical CNNs [16], [17], [18] were originally designed for image classification tasks.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [16] Krizhevsky A., Sutskever I., and Hinton G. E., “ ImageNet classification with deep convolutional neural networks,” in Proc. 25th Int. Conf. Neural Inf. Process. Syst., 2012, pp. 1097– 1105.

      Not checked in the free run.

    • cites [17] Simonyan K. and Zisserman A., “ Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556. source

      Not checked in the free run.

    • cites [18] He K., Zhang X., Ren S., and Sun J., “ Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 770– 778. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Machine learning methods for interactive segmentation can achieve higher segmentation accuracy with fewer user interactions, but are limited by hand-crafted features that depend on the user's experience.

    “Machine learning methods [6], [37], [38] have been proposed to learn from user interactions. They can achieve higher segmentation accuracy with fewer user interactions. However, they are limited by hand-crafted features that depend on the user's experience.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [6] Wang G., Zuluaga M. A., Pratt R., Aertsen M., Doel T., Klusmann M., David A. L., Deprest J., Vercauteren T., and Ourselin S., “ Slic-Seg: A minimally interactive segmentation of the placenta from sparse and motion-corrupted fetal MRI in multiple views,” Med. Image Anal., vol. 34, pp. 137– 147, 2016. 27179367 PMC5052128 source

      Not checked in the free run.

    • cites [37] Barinova O., Shapovalov R., Sudakov S., and Velizhev A., “ Online random forest for interactive image segmentation,” in Proc. Int. Workshop Exp. Economics Mach. Learn., 2012, pp. 1– 8.

      Not checked in the free run.

    • cites [38] Luengo I., Darrow M. C., Spink M. C., Sun Y., Dai W., He C. Y., Chiu W., Pridmore T., Ashton A. W., Duke E. M., Basham M., and French A. P., “ SuRVoS: Super-region volume segmentation workbench,” J. Structural Biol., vol. 198, no. 1, pp. 43– 53, 2017. PMC5405849 28246039 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Early patch or region-based CNN methods achieved higher accuracy than traditional methods that relied on hand-crafted features, but suffered from inefficiency for testing.

    “Some early works adapted such networks for pixel labeling with patch or region-based methods [19], [20]. Such methods achieved higher accuracy than traditional methods that relied on hand-crafted features. However, they suffered from inefficiency for testing.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [19] Havaei M., Davy A., Warde-Farley D., Biard A., Courville A., Bengio Y., Pal C., Jodoin P.-M., and Larochelle H., “ Brain tumor segmentation with deep neural networks,” Med. Image Anal., vol. 35, pp. 18– 31, 2016. 27310171 source

      Not checked in the free run.

    • cites [20] Girshick R., Donahue J., Darrell T., and Malik J., “ Rich feature hierarchies for accurate object detection and semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2014, pp. 580– 587. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    GrabCut works with a user-provided bounding box and requires fewer interactions than Graph Cuts, but its performance is limited by the representativity of the underpinning GMM.

    “GrabCut [9] works with a user-provided bounding box to start the segmentation and requires fewer interactions compared with Graph Cuts [8], but the performance is still limited by the representativity of the underpinning GMM.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [9] Rother C., Kolmogorov V., and Blake A., “ GrabCut: Interactive foreground extraction using iterated graph cuts,” ACM Trans. Graph., vol. 23, no. 3, pp. 309– 314, 2004. source

      Not checked in the free run.

    • cites [8] Boykov Y. Y. and Jolly M. P., “ Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images,” in Proc. 8th IEEE Int. Conf. Comput. Vis., 2001, pp. 105– 112. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    ScribbleSup trains CNNs for semantic segmentation supervised by scribbles.

    “ScribbleSup [39] also trains CNNs for semantic segmentation supervised by scribbles.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [39] Lin D., Dai J., Jia J., He K., and Sun J., “ ScribbleSup: Scribble-supervised convolutional networks for semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 3159– 3167. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    FCNs take an entire image as input and give a dense segmentation, using deconvolution/upsampling to overcome loss of spatial resolution.

    “FCNs [11] take an entire image as input and give a dense segmentation. In order to overcome the problem of loss of spatial resolution due to multi-stage max-pooling and downsampling, it uses a stack of deconvolution (a.k.a. upsampling) layers and activation functions to upsample the feature maps.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [11] Long J., Shelhamer E., and Darrell T., “ Fully convolutional networks for semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2015, pp. 3431– 3440. 27244717

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    CNNs have not yet been adapted to interactive medical image segmentation.

    “However, they have not yet been adapted to interactive medical image segmentation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Dilated convolution preserves the resolution of feature maps and enlarges the receptive field to incorporate larger contextual information.

    “To overcome the drawbacks of successive max-pooling and downsampling that lead to a loss of feature map resolution, dilated convolution [22], [23] was proposed to preserve the resolution of feature maps and enlarge the receptive field to incorporate larger contextual information.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [22] Chen L.-C., Papandreou G., Kokkinos I., Murphy K., and Yuille A. L., “ Semantic image segmentation with deep convolutional nets and fully connected CRFs,” arXiv preprint arXiv:1412.7062. 28463186 source

      Not checked in the free run.

    • cites [23] Yu F. and Koltun V., “ Multi-scale context aggregation by dilated convolutions,” arXiv preprint arXiv:1511.07122. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Multi-scale features extracted from CNNs have been shown to be effective for improving segmentation accuracy.

    “Multi-scale features extracted from CNNs have been shown to be effective for improving segmentation accuracy [11], [22], [23].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [11] Long J., Shelhamer E., and Darrell T., “ Fully convolutional networks for semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2015, pp. 3431– 3440. 27244717 source

      Not checked in the free run.

    • cites [22] Chen L.-C., Papandreou G., Kokkinos I., Murphy K., and Yuille A. L., “ Semantic image segmentation with deep convolutional nets and fully connected CRFs,” arXiv preprint arXiv:1412.7062. 28463186 source

      Not checked in the free run.

    • cites [23] Yu F. and Koltun V., “ Multi-scale context aggregation by dilated convolutions,” arXiv preprint arXiv:1511.07122. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The geodesic distance transform encodes spatial regularization and contrast-sensitivity.

    “the geodesic distance transform [35], [36], [41] encodes spatial regularization and contrast-sensitivity”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [35] Bai X. and Sapiro G., “ A geodesic framework for fast interactive image and video segmentation and matting,” in Proc. IEEE 11th Int. Conf. Comput. Vis., 2007, pp. 1– 8. source

      Not checked in the free run.

    • cites [36] Criminisi A., Sharp T., and Blake A., “ GeoS: Geodesic image segmentation,” in Proc. Eur. Conf. Comput. Vis., 2008, pp. 99– 112. source

      Not checked in the free run.

    • cites [41] Kontschieder P., Kohli P., Shotton J., and Criminisi A., “ GeoF geodesic forests for learning coupled predictors,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2013, pp. 65– 72. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Most interactive image segmentation methods rely on low-level features and require a relatively large amount of user interactions to deal with images with low contrast and ambiguous boundaries.

    “However, most of these methods rely on low-level features and require a relatively large amount of user interactions to deal with images with low contrast and ambiguous boundaries.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [8] Boykov Y. Y. and Jolly M. P., “ Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images,” in Proc. 8th IEEE Int. Conf. Comput. Vis., 2001, pp. 105– 112. source

      Not checked in the free run.

    • cites [35] Bai X. and Sapiro G., “ A geodesic framework for fast interactive image and video segmentation and matting,” in Proc. IEEE 11th Int. Conf. Comput. Vis., 2007, pp. 1– 8. source

      Not checked in the free run.

    • cites [7] Grady L., “ Random walks for image segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 11, pp. 1768– 1783, Nov. 2006. 17063682 source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Random Walks and Graph Cuts learn from scribbles and allow additional scribbles for refinement, but need a large number of scribbles to get satisfactory segmentation.

    “Random Walks [7] and Graph Cuts [8] learn from scribbles and allow the user to provide additional scribbles for refinement. They use Random Walker and Gaussian Mixture Model (GMM) as the underpinning model. However, they need a large number of scribbles to get satisfactory segmentation.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [7] Grady L., “ Random walks for image segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 11, pp. 1768– 1783, Nov. 2006. 17063682 source

      Not checked in the free run.

    • cites [8] Boykov Y. Y. and Jolly M. P., “ Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images,” in Proc. 8th IEEE Int. Conf. Comput. Vis., 2001, pp. 105– 112. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    CRFs have been widely used to enhance segmentation accuracy by introducing spatial consistency.

    “Graphical models such as CRFs [22], [42], [43] have been widely used to enhance segmentation accuracy by introducing spatial consistency.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [22] Chen L.-C., Papandreou G., Kokkinos I., Murphy K., and Yuille A. L., “ Semantic image segmentation with deep convolutional nets and fully connected CRFs,” arXiv preprint arXiv:1412.7062. 28463186 source

      Not checked in the free run.

    • cites [42] Boykov Y. and Kolmogorov V., “ An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 9, pp. 1124– 1137, Sep. 2004. 15742889 source

      Not checked in the free run.

    • cites [43] Yuan J., Bae E., and Tai X. C., “ A study on continuous max-flow and min-cut approaches,” in Proc. IEEE Comput. Society Conf. Comput. Vis. Pattern Recognit., 2010, pp. 2217– 2224. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Most existing interactive segmentation methods require a large amount of user interactions and long user time, or have limited learning ability with their underpinning models.

    “Despite the large number of existing interactive segmentation methods [3], most of them are confronted by requiring a large amount of user interactions and long user time, or limited learning ability with their underpinning models.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [3] Zhao F. and Xie X., “ An overview of interactive medical image segmentation,” Ann. BMVA, vol. 2013, no. 7, pp. 1– 22, 2013.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    3D U-Net learns from sparsely annotated images and can be used for semi-automatic segmentation.

    “For instance, 3D U-Net [21] learns from sparsely annotated images and can be used for semi-automatic segmentation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] Abdulkadir A., Lienkamp S. S., Brox T., and Ronneberger O., “ 3D U-Net: Learning dense volumetric segmentation from sparse annotation,” in Proc. Medical Image Comput. Comput.-Assisted Intervention, 2016, pp. 424– 432. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Mean-field approximation is often used for efficient inference of the CRF while allowing learning parameters by back-propagation.

    “Alternatively, mean-field approximation [45], [50], [52] is often used for efficient inference of the CRF while allowing learning parameters by back-propagation.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [45] Krähenbühl P. and Koltun V., “ Efficient inference in fully connected CRFs with gaussian edge potentials,” in Proc. Neural Inf. Process. Syst. Conf., 2011, pp. 109– 117. source

      Not checked in the free run.

    • cites [50] Zheng S., Jayasumana S., Romera-Paredes B., Vineet V., Su Z., Du D., Huang C., and Torr P. H. S., “ Conditional random fields as recurrent neural networks,” in Proc. IEEE Int. Conf. Comput. Vis., 2015, pp. 1529– 1537. source

      Not checked in the free run.

    • cites [52] Vemulapalli R., Tuzel O., Liu M.-y., and Chellappa R., “ Gaussian conditional random field network for semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 3224– 3233. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The same initial segmentation and the same set of user interactions were used by the five refinement methods.

    “in which the same initial segmentation and the same set of user interactions were used by the five refinement methods.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In [52], a Gaussian Mean Field network was proposed and combined with CNNs where all parameters are trainable.

    “In [52], a Gaussian Mean Field (GMF) network was proposed and combined with CNNs where all the parameters are trainable.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [52] Vemulapalli R., Tuzel O., Liu M.-y., and Chellappa R., “ Gaussian conditional random field network for semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 3224– 3233. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In [54], a generic CNN-CRF model was proposed to handle arbitrary potentials for labeling body parts in depth images, but it has not yet been validated with other segmentation applications.

    “In [54], a generic CNN-CRF model was proposed to handle arbitrary potentials for labeling body parts in depth images. However, it has not yet been validated with other segmentation applications.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [54] Kirillov A., Zheng S., Schlesinger D., Forkel W., Zelenin A., Torr P., and Rother C., “ Efficient likelihood learning of a generic CNN-CRF model for semantic segmentation,” arXiv preprint arXiv:1511.05067. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    GeoF uses the geodesic distance to encode variable dependencies in the feature space and is combined with Random Forests for semantic segmentation, but it is not designed to deal with user interactions.

    “GeoF [41] uses the geodesic distance to encode variable dependencies in the feature space and it is combined with Random Forests for semantic segmentation. However, it is not designed to deal with user interactions.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [41] Kontschieder P., Kohli P., Shotton J., and Criminisi A., “ GeoF geodesic forests for learning coupled predictors,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2013, pp. 65– 72. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    More freeform pairwise potentials for a pair of super-pixels or image patches were proposed, but such CRFs have a low resolution.

    “More freeform pairwise potentials for a pair of super-pixels or image patches were proposed in [27], [53], but such CRFs have a low resolution.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [27] Lin G., Shen C., Reid I., and van dan Hengel A., “ Efficient piecewise training of deep structured models for semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 3194– 3203. source

      Not checked in the free run.

    • cites [53] Liu F., Shen C., and Lin G., “ Deep convolutional neural fields for depth estimation from a single image,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2014, pp. 5162– 5170. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In CRF-RNN, pair-wise potentials are limited to weighted Gaussians and not all parameters are trainable due to the Permutohedral lattice implementation.

    “However, the pair-wise potentials in [50] are limited to weighted Gaussians and not all the parameters are trainable due to the Permutohedral lattice implementation [51].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [50] Zheng S., Jayasumana S., Romera-Paredes B., Vineet V., Su Z., Du D., Huang C., and Torr P. H. S., “ Conditional random fields as recurrent neural networks,” in Proc. IEEE Int. Conf. Comput. Vis., 2015, pp. 1529– 1537. source

      Not checked in the free run.

    • cites [51] Adams A., Baek J., and Davis M. A., “ Fast high-dimensional filtering using the permutohedral lattice,” Comput. Graph. Forum, vol. 29, no. 2, pp. 753– 762, 2010. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The ground truth for 2D placenta segmentation was manually delineated by an experienced Radiologist.

    “The ground truth was manually delineated by an experienced Radiologist.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The 2D segmentation results can also be used for motion correction and high-resolution volume reconstruction.

    “The 2D segmentation results can also be used for motion correction and high-resolution volume reconstruction [63].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [63] Keraudren K., Kuklisova-Murgasova M., Kyriakopoulou V., Malamateniou C., Rutherford M. A., Kainz B., Hajnal J. V., and Rueckert D., “ Automated fetal brain segmentation from 2D MRI slices for motion correction,” NeuroImage, vol. 101, pp. 633– 643, 2014. 25058899 source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    3D P-Net is more memory efficient and can perform inference on a 3D volume in interactive time.

    “It is more memory efficient and therefore can perform inference on a 3D volume in interactive time.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    R-Net is only trained with user interactions once since it takes a considerable time to re-train a CNN model with a large training set.

    “Differently from previous works [55] that re-train the learning model each time when new user interactions are given, the proposed R-Net is only trained with user interactions once since it takes a considerable time to re-train a CNN model with a large training set.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [55] Wang G., Zuluaga M. A., Pratt R., Aertsen M., Doel T., Klusmann M., David A. L., Deprest J., Vercauteren T., and Ourselin S., “ Dynamically balanced online random forests for interactive scribble-based segmentation,” in Proc. Medical Image Comput. Comput.-Assisted Intervention, 2016, pp. 352– 360. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Continuous max-flow can also be used for the minimization, but its parameters are manually designed.

    “Continuous max-flow [43] can also be used for the minimization, but its parameters are manually designed.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [43] Yuan J., Bae E., and Tai X. C., “ A study on continuous max-flow and min-cut approaches,” in Proc. IEEE Comput. Society Conf. Comput. Vis. Pattern Recognit., 2010, pp. 2217– 2224. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Two users (an Obstetrician and a Radiologist) respectively used the interactive methods to segment every test image until the result was visually acceptable.

    “Two users (an Obstetrician and a Radiologist) respectively used these interactive methods to segment every test image until the result was visually acceptable.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    In [50], a CRF based on RNN was proposed and it can be trained by back-propagation.

    “In [50], a CRF based on RNN was proposed and it can be trained by back-propagation.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [50] Zheng S., Jayasumana S., Romera-Paredes B., Vineet V., Su Z., Du D., Huang C., and Torr P. H. S., “ Conditional random fields as recurrent neural networks,” in Proc. IEEE Int. Conf. Comput. Vis., 2015, pp. 1529– 1537. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The CRF-RNN network formulates dense CRFs as RNNs so that CNNs and CRFs can be jointly trained in an end-to-end system for segmentation.

    “The CRF-RNN network [50] formulated dense CRFs as RNNs so that the CNNs and CRFs can be jointly trained in an end-to-end system for segmentation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [50] Zheng S., Jayasumana S., Romera-Paredes B., Vineet V., Su Z., Du D., Huang C., and Torr P. H. S., “ Conditional random fields as recurrent neural networks,” in Proc. IEEE Int. Conf. Comput. Vis., 2015, pp. 1529– 1537. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given

    P-Net and R-Net use a resolution-preserving structure that captures high-level features from a large receptive field without loss of resolution.

    “P-Net and R-Net use a resolution-preserving structure that captures high-level features from a large receptive field without loss of resolution.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Long-distance dependency helps to improve segmentation in most RGB images.

    “Second, though long-distance dependency helps to improve segmentation in most RGB images [22], [45], [50]”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [22] Chen L.-C., Papandreou G., Kokkinos I., Murphy K., and Yuille A. L., “ Semantic image segmentation with deep convolutional nets and fully connected CRFs,” arXiv preprint arXiv:1412.7062. 28463186 source

      Not checked in the free run.

    • cites [45] Krähenbühl P. and Koltun V., “ Efficient inference in fully connected CRFs with gaussian edge potentials,” in Proc. Neural Inf. Process. Syst. Conf., 2011, pp. 109– 117. source

      Not checked in the free run.

    • cites [50] Zheng S., Jayasumana S., Romera-Paredes B., Vineet V., Su Z., Du D., Huang C., and Torr P. H. S., “ Conditional random fields as recurrent neural networks,” in Proc. IEEE Int. Conf. Comput. Vis., 2015, pp. 1529– 1537. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The raster-scan algorithm is fast due to accessing the image memory in contiguous blocks.

    “It is fast due to accessing the image memory in contiguous blocks.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [36] Criminisi A., Sharp T., and Blake A., “ GeoS: Geodesic image segmentation,” in Proc. Eur. Conf. Comput. Vis., 2008, pp. 99– 112.

      Not checked in the free run.

54 of 54

  • The paper relies on this without support.

    Poor image quality, large variations among patients, inhomogeneous appearances from pathology, and variability of protocols among clinicians lead to limitations of automatic segmentation.

    “This is typically due to poor image quality (with noise, partial volume effect, artifacts and low contrast), large variations among patients, inhomogeneous appearances brought by pathology, and variability of protocols among clinicians leading to different definitions of a given structure boundary.”

  • The paper relies on this without support.

    Providing manual annotations for segmentation is time-consuming and labor-intensive.

    “As providing manual annotations for segmentation is time-consuming and labor-intensive, an efficient interactive segmentation tool is of great importance for practical use.”

  • The paper relies on this without support.

    The initial automatic segmentation from P-Net is high-quality.

    “the method starts with a high-quality automatic segmentation”

  • The paper relies on this without support.

    Simulated user interactions (random clicks in mis-segmented regions) are representative of real user interactions for training R-Net.

    “we automatically simulate user interactions to train R-Net with CRF-Net(fu)”

  • The paper relies on this without support.

    The manually delineated ground truth is accurate.

    “The ground truth was manually delineated by an experienced Radiologist.”

  • The paper relies on this without support.

    Automatic brain tumor segmentation remains technically challenging because tumor size, shape, and localization vary among patients and boundaries between adjacent structures are often ambiguous.

    “automatic brain tumor segmentation remains technically challenging because 1) the size, shape, and localization of brain tumors have considerable variations among patients; 2) the boundaries between adjacent structures are often ambiguous.”

  • The paper relies on this without support.

    Segmentation of brain tumors can provide better volumetric measurements and therefore has enormous potential value for improved diagnosis, treatment planning, and follow-up of individual patients.

    “Segmentation of brain tumors can provide better volumetric measurements and therefore has enormous potential value for improved diagnosis, treatment planning, and follow-up of individual patients.”
7 of 7

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Semi- or fully automatic segmentation methods can significantly reduce the time and labor required, increase consistency, and enable the analysis of large-scale datasets.

    “Semi- or fully automatic segmentation methods can significantly reduce the time and labor required, increase consistency, and enable the analysis of large-scale datasets4.”

    From Segment anything in medical images (2024) · cites this paper as [4]

    The Introduction (page unavailable) says manual segmentation is time-consuming and labor-intensive, motivating efficient interactive methods; the Method describes an automatic initial segmentation intended to minimize user interactions. This supports the time-and-labor aspect, but our copy does not establish that semi- or fully automatic methods increase consistency or enable analysis of large-scale datasets. The Introduction also cautions that fully automatic methods rarely achieve sufficient accuracy and robustness for clinical use. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Existing deep learning-based interactive segmentation methods focus mainly on limited tasks and image modalities.

    “while existing deep learning-based interactive segmentation methods focus mainly on limited tasks and image modalities.”

    From Segment anything in medical images (2024) · cites this paper as [4]

    The Related Works section describes deep-learning approaches for specific tasks, including semantic segmentation, fetal MRI segmentation, and interactive object selection. It does not establish that existing methods mainly focus on a limited range of tasks or image modalities; its stated criticism is that several methods do not allow interactive refinement at test time. (LLM, conf=0.74)

    Experimental — accuracy not yet measured

  • In annotation workflows, an interactive model can reduce the amount of slice-by-slice contouring needed to create training labels.

    “In annotation workflows, an interactive model can reduce the amount of slice-by-slice contouring needed to create training labels [28,31].”

    From Correction-aware interactive 3D tumor segmentation with sparse and revisable prompts (2026) · cites this paper as [28]

    The Introduction and Method describe an interactive model that starts with automatic segmentation and uses clicks or short scribbles to refine it, reducing manual annotation effort. They do not establish that it reduces slice-by-slice contouring specifically to create training labels; the Method instead describes using existing ground truth to train the refinement network. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

Cites

Flagged first, then by citations

16 of the 90 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. payet, 2010, (RF)^2 &#x2013; random forest random field, Proc Neural Inf Process Syst Conf, 1885

Cited by

Sorted by citations

Related

Authors
Wang, G., Zuluaga, M. A., Li, W., Pratt, R., Patel, P. A., Aertsen, M., Doel, T., David, A. L., Deprest, J., Ourselin, S., & Vercauteren, T.
Affiliations
  • Wellcome / EPSRC Centre for Interventional and Surgical Sciences (GB)
  • University College London (GB)
  • KU Leuven (BE)
Corresponding author
Wang, Guotai
ORCID
Venue
IEEE Transactions on Pattern Analysis and Machine Intelligence
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2019-07-01
Volume
41
Issue
7
Pages
1559-1572
ISSN
  • 0162-8828 (ISSN-L)
  • 2160-9292
  • 1939-3539
DOI
10.1109/tpami.2018.2840695
PMC
PMC6594450
OpenAlex
W2732931556
Semantic Scholar
d66c118307ca332e24b1d53bcf152563e3c48b6b
Cited by
502
References
90
Citations by year
2026: 47 · 2025: 74 · 2024: 64 · 2023: 75 · 2022: 76 · 2021: 82 · 2020: 42 · 2019: 31 · 2018: 11
FWCI
20.76
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Lirias (submittedVersion, other-oa)
  • UCL Discovery (University College London) (submittedVersion)
  • Research Portal (King's College London) (submittedVersion, cc-by)
  • arXiv (Cornell University)
Funders
  • Wellcome Trust (WT101957)
  • Engineering and Physical Sciences Research Council (NS/A000027/1, EP/H046410/1, EP/J020990/1, EP/K005278)
  • Wellcome/EPSRC (203145Z/16/Z)
  • National Institute for Health Research (NIHR BRC UCLH/UCL)
  • Royal Society (RG160569)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Life Sciences › Neuroscience › Neurology › Brain Tumor Detection and Classification
Keywords
interactive segmentation, medical image segmentation, user interaction, conditional random fields, placenta segmentation, fetal MRI, brain tumor segmentation, FLAIR MRI, dense prediction, segmentation refinement
MeSH terms
Brain Neoplasms, Female, Humans, Image Interpretation, Computer-Assisted, Magnetic Resonance Imaging, Placenta, Pregnancy, Neural Networks, Computer, Imaging, Three-Dimensional

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-08.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-08.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review