CoChat
Sign Up
Cited by 758Medical Image Segmentation TechniquesAdvanced Neural Network ApplicationsArtificial Intelligence in Healthcare and Education

Anatomically constrained neural networks (ACNNS): application to cardiac image enhancement and segmentation

Open access
Oktay, Ozan, Ferrante, Enzo, Kamnitsas, Konstantinos, Heinrich, Mattias, Bai, Wenjia, Caballero, Jose, Cook, Stuart A., de Marvao, Antonio +5 more·2018·IEEE Transactions on Medical Imaging
DOI 10.1109/tmi.2017.2743464
Open this paper in CoChat — free account

Abstract

Incorporation of prior knowledge about organ shape and location is key to improve performance of image analysis approaches. In particular, priors can be useful in cases where images are corrupted and contain artefacts due to limitations in image acquisition. The highly constrained nature of anatomical objects can be well captured with learning-based techniques. However, in most recent and promising techniques such as CNN-based segmentation it is not obvious how to incorporate such prior knowledge. State-of-the-art methods operate as pixel-wise classifiers where the training objectives do not incorporate the structure and inter-dependencies of the output. To overcome this limitation, we propose a generic training strategy that incorporates anatomical prior knowledge into CNNs through a new regularisation model, which is trained end-to-end. The new framework encourages models to follow the global anatomical properties of the underlying anatomy (e.g. shape, label structure) via learnt non-linear representations of the shape. We show that the proposed approach can be easily adapted to different analysis tasks (e.g. image enhancement, segmentation) and improve the prediction accuracy of the state-of-the-art models. The applicability of our approach is shown on multi-modal cardiac data sets and public benchmarks. In addition, we demonstrate how the learnt deep models of 3-D shapes can be interpreted and used as biomarkers for classification of cardiac pathologies.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper proposes anatomically constrained neural networks (ACNN), a generic end-to-end training strategy that uses a stacked convolutional autoencoder and T-L network to impose learnt anatomical shape priors on CNN predictions. It demonstrates improved cardiac MR segmentation, 3D ultrasound segmentation, MR super-resolution, and pathology classification over state-of-the-art baselines.

Key findings

  • For cardiac cine-MR segmentation, ACNN-Seg improved endocardium Dice to .939 ± .017 versus .908 ± .021 for 2D-FCN, and myocardium Dice to .811 ± .027 versus .727 ± .046; mean surface distances were also lower (endocardium 1.37 ± 0.42 mm vs 2.07 ± 0.61 mm; myocardium 1.14 ± 0.22 mm vs 1.58 ± 0.44 mm), with p < 0.001 for Dice and mean distance.
  • On the CETUS’14 3D-US benchmark, ACNN-Seg achieved end-diastole Dice .912 ± .023 versus .906 ± .026 for FCN and .894 ± .041 for BEAS, and end-systole Dice .873 ± .051 versus .872 ± .050 for FCN and .856 ± .057 for BEAS; Hausdorff distance at ED was 6.96 ± 1.75 mm versus 11.94 ± 9.46 mm for FCN.
  • For cardiac MR super-resolution, ACNN-SR achieved SSIM .796 ± .041 versus .783 ± .046 for SR-CNN, clinician MOS scores of 4.36 ± 0.62 and 4.25 ± .68 versus 3.59 ± 1.05 and 3.85 ± .70, and inference time .06 s versus .29 s.
  • For pathology classification, T-L codes gave 83.3% accuracy from ED phase and 91.6% with ED+ES, compared with PCA codes at 76.6% and 86.6%, using 10-fold cross-validation on 60 CMR sequences.

Methods

  • ACNN adds a shape-regularisation loss to standard CNN training: segmentation uses cross-entropy plus a low-dimensional shape loss and weight decay, while super-resolution uses smooth ℓ1 reconstruction plus a global shape loss and weight decay. The regulariser is a stacked convolutional autoencoder and T-L network that learns 64-dimensional anatomical shape/label representations; it is used only during training, not inference.
  • Data include the UK Digital Heart Project dataset of 1200 cine 2D stack and 3D HR cardiac MR image pairs, split into 900 training, 100 validation, and 200 testing for segmentation, and 1000 training / 200 testing for super-resolution; CETUS’14 with 45 3D+time ultrasound sequences from 15 healthy subjects and 30 patients, evaluated blinded on 30 test sequences; and ACDC MICCAI’17 with 20 cine stacks per class for classification.
  • Comparisons include 2D-FCN, 3D-UNet, AE-Seg, 3D-Seg, BEAS, FCN, and SR-CNN; metrics include Dice, mean and Hausdorff surface distances, SSIM, mean opinion score, ejection-fraction correlation/bias, and classification accuracy, with Wilcoxon signed-rank tests for top methods.

Limitations

  • The paper states that 3D-US training data were limited to 15 image sequences; segmentation results could improve by segmenting ED and ES frames simultaneously or using temporal information such as optical flow; classification accuracy could improve by training the AE and T-L models with a classification objective; and achieving the best possible segmentation accuracy was not the main focus.
  • The paper also notes that the T-L regulariser is used only at training time, that latent codes are expected to be correlated because the decoder merges codes across spatial dimensions, and that a variational autoencoder was not used because the K-L term would reduce representation power and generative sampling was not needed.
  • Evident from the text: the method is evaluated on cardiac applications only, and the authors state future work will extend ACNN to tasks such as human pose estimation and landmark localisation.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/tmi.2017.2743464

Cites

53 of 53 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 34 cited sources checked · 0 supported · 0 flagged

  • Findingown data
    § B. Contributions · p. 3

    The evaluation shows an implicit statistical parametrisation of the left ventricular shape via NNs for pathology classification.

    “(II) An implicit statistical parametrisation of the left ventricular shape via NNs for pathology classification;”
    Quote in the paperSubstantial claim
  • Findingown data
    § Anatomically Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement and Segmentation · p. 1

    The proposed approach is applicable to multi-modal cardiac datasets and public benchmarks.

    “The applicability of our approach is shown on multi-modal cardiac datasets and public benchmarks.”
    Quote in the paperSubstantial claim
  • Findingown data
    § B. Contributions · p. 3

    The evaluation shows a sub-pixel cardiac MR image segmentation approach that, in contrast to previous CNN approaches, is robust against slice misalignment and coverage problems.

    “Our evaluation shows: (I) A sub-pixel cardiac MR image segmentation approach that, in contrast to previous CNN approaches [44], [2], is robust against slice misalignment and coverage problems;”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [44] [44] P. V. Tran. A fully convolutional neural network for cardiac segmentation in short-axis MRI. arXiv preprint arXiv:1604.00494 , 2016. - source

      Not checked in the free run.

    • cites [2] [2] M. Avendi, A. Kheradvar, and H. Jafarkhani. A combined deep-learning and deformable-model approach to fully automatic segmentation of the left ventricle in cardiac MRI. MedIA , 30:108-119, 2016. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Findingown data
    § Anatomically Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement and Segmentation · p. 1

    The proposed approach improves the prediction accuracy of state-of-the-art models.

    “We show that the proposed approach can be easily adapted to different analysis tasks ( e.g. image enhancement, segmentation) and improve the prediction accuracy of the state-of-the-art models.”
    Quote in the paperSubstantial claim
  • Findingown data
    § D. Cardiac 3D Ultrasound Image Segmentation · p. 9

    The ACNN model improves shape delineation of the LV over standard FCN approaches, as seen in distance error metrics.

    “The main contribution of ACNN model over the standard FCN approaches is the improved shape delineation of the LV, as it can be seen in terms of the distance error metrics.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The CETUS web platform evaluation is a valid blinded benchmark for segmentation accuracy.
  • May combine two claims. 34
  • The quote only partly states this claim. 5
  • Classified as probably not a substantial claim. 16

  • May combine two claims.

    Findingown data
    § A. Clinical Motivation · p. 2

    The proposed segmentation and SR models become more robust against imaging artefacts, as shown by state-of-the-art results on the MICCAI'14 CETUS public benchmark.

    “We show that the proposed segmentation and SR models become more robust against imaging artefacts mentioned earlier which is underlined by our state-of-the-art results on the MICCAI'14 CETUS public benchmark [8].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [8] [8] O. Bernard, J. G. Bosch, B. Heyde, M. Alessandrini, D. Barbosa, S. Camarasu-Pop, F. Cervenansky, S. Valette, O. Mirea, M. Bernier, et al. Standardized evaluation system for left ventricular segmentation algorithms in 3D echocardiography. IEEE TMI , 35(4):967-977, 2016. - source

      Not checked in the free run.

  • May combine two claims.

    Findingnone given
    § B. Contributions · p. 3

    The proposed image SR technique extends previous work and is robust against slice misalignments.

    “(III) An image SR technique that extends previous work [34] and that is robust against slice misalignments;”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [34] [34] O. Oktay, W. Bai, M. Lee, R. Guerrero, K. Kamnitsas, J. Caballero, A. de Marvao, S. Cook, D. ORegan, and D. Rueckert. Multi-input cardiac image super-resolution using convolutional neural networks. In International Conference on MICCAI , pages 246-254. Springer, 2016. - source

      Not checked in the free run.

  • May combine two claims.

    Findingreasoning
    § B. Contributions · p. 3

    The proposed SR approach is computationally more efficient than the state-of-the-art SR-CNN model because feature extraction is performed in the low-dimensional image space.

    “our approach is computationally more efficient than the state-of-the-art SR-CNN model [34] as the feature extraction is performed in the low-dimensional image space.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [34] [34] O. Oktay, W. Bai, M. Lee, R. Guerrero, K. Kamnitsas, J. Caballero, A. de Marvao, S. Cook, D. ORegan, and D. Rueckert. Multi-input cardiac image super-resolution using convolutional neural networks. In International Conference on MICCAI , pages 246-254. Springer, 2016. - source

      Not checked in the free run.

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § A. Clinical Datasets · p. 7

    On 2D cardiac MR stacks, ACNN-Seg achieved lower mean surface distance and higher Dice scores for endocardium and myocardium than baseline methods, with statistically significant improvements for endocardium mean distance, endocardium Dice, myocardium mean distance, and myocardium Dice (p < 0.001).

    “ACNN-Seg | 1.37 ± 0.42 | 7.89 ± 3.83 | .939 ± .017 | 1.14 ± 0.22 | 7.31 ± 3.59 | .811 ± .027 | 1 . 60 × 10 6 |”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The high-resolution motion-free cardiac MR images provide an appropriate reference for evaluating segmentation performance on low-resolution 2D stacks.

  • May combine two claims.

    Findingown data
    § E. Learnt Hidden Representations · p. 6

    The learnt latent representations of a T-L model trained with cardiac MR segmentations generally follow a normal distribution and are not separated into multiple clusters.

    “It is observed that the learnt latent representations in general follow a normal distribution and they are not separated in multi-clusters ( e.g. mixture of Gaussians).”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingreasoning
    § D. Cardiac 3D Ultrasound Image Segmentation · p. 9

    Hausdorff distances were reduced significantly by the ACNN approach.

    “In particular, Hausdorff distances were reduced significantly”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The CETUS web platform evaluation is a valid blinded benchmark for segmentation accuracy.

  • May combine two claims.

    Findingown data
    § C. Cardiac Cine-MR Image Segmentation · p. 8

    Previous slice-by-slice segmentation approaches validated on LR annotations produce label maps far off from the true underlying ventricular geometry, which can limit analysis of ventricle morphology.

    “Previous slice by slice segmentation approaches validated their methods on LR annotations; however, we see that the produced label maps are far off from the true underlying ventricular geometry and it can be a limiting factor for the analysis of ventricle morphology.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HR ground-truth label maps projected from HR image space are an accurate reference for segmentation evaluation.

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § D. Cardiac 3D Ultrasound Image Segmentation · p. 9

    The ACNN approach improves normalised Dice score over FCN, but the improvement is not statistically significant due to the large volumetric size of the LV cavity.

    “Similarly, we observed an improvement in terms of normalised Dice score, which was quantitatively not significant due to large volumetric size of the LV cavity.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The CETUS web platform evaluation is a valid blinded benchmark for segmentation accuracy.

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § D. Cardiac 3D Ultrasound Image Segmentation · p. 9

    The ACNN approach ensures better consistency between ED and ES frames than compared methods, although none of the methods used temporal information.

    “It is observed that the ACNN approach ensures better consistency between frames although none of the methods have used temporal information.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The CETUS web platform evaluation is a valid blinded benchmark for segmentation accuracy.

  • May combine two claims.

    Findingown data
    § C. Cardiac Cine-MR Image Segmentation · p. 8

    The small-capacity AE-Seg model (8 × 10^4 parameters) is not suitable for cardiac image segmentation because the second stage in the cascaded model does not improve performance significantly.

    “We also analysed the performance change in AE-Seg with respect to the number of parameters, which shows that the small capacity AE-Seg model ( 8 × 10 4 pars) is not suitable for cardiac image segmentation as the second stage in the cascaded model does not improve the performance significantly.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HR ground-truth label maps projected from HR image space are an accurate reference for segmentation evaluation.

  • May combine two claims.

    Findingown data
    § C. Cardiac Cine-MR Image Segmentation · p. 8

    Reconstruction of label maps from low-dimensional representations using only the T-L network was limited; ventricle boundaries were not delineated properly and a rough segmentation was generated (DSC .734).

    “We observed that reconstruction of label-maps from low dimensional representations was limited since the ventricle boundaries were not delineated properly but rather a rough segmentation was generated (DSC: . 734 ).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HR ground-truth label maps projected from HR image space are an accurate reference for segmentation evaluation.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data
    § VI. SUPPLEMENTARY MATERIAL · p. 13

    The Sorensen-Dice loss tested as an alternative to cross-entropy loss yielded degraded performance because cross-entropy is a smoother function.

    “In the segmentation problem, the Sorensen-Dice loss was tested in the experiments as an alternative to the cross-entropy loss, yet we observed a degraded performance since the latter is a smoother function.”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § F. Learnt Latent Representations and Pathology Classification · p. 10

    Including codes from the ES phase improved classification accuracies to 86.6% with PCA codes and 91.6% with T-L codes.

    “By including the codes from ES phase, the classification accuracies were improved to 86 . 6% vs 91 . 6% .”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HR ground-truth label maps projected from HR image space are an accurate reference for segmentation evaluation.

  • May combine two claims.

    Findingown data
    § F. Learnt Latent Representations and Pathology Classification · p. 10

    Using 10-fold cross-validation on 60 CMR sequences, classification accuracy was 76.6% with PCA codes and 83.3% with T-L codes extracted from the ED phase.

    “We used 10 -fold cross-validation on 60 CMR sequences and obtained 76 . 6% vs 83 . 3% accuracy using PCA and T-L codes extracted from ED phase.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The HR ground-truth label maps projected from HR image space are an accurate reference for segmentation evaluation.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given
    § D. Cardiac 3D Ultrasound Image Segmentation · p. 9

    The fully convolutional network model [11] is more memory efficient compared to the standard 3D-UNet architecture [12].

    “It is also observed to be more memory efficient compared to the standard 3D-UNet architecture [12].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [11] [11] H. Chen, Y. Zheng, J.-H. Park, P.-A. Heng, and S. K. Zhou. Iterative multi-domain regularized deep learning for anatomical structure detection and segmentation from ultrasound images. In International Conference on MICCAI , pages 487-495. Springer, 2016. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [12] [12] ¨ O. C ¸ ic ¸ek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger. 3D U-Net: Learning dense volumetric segmentation from sparse annotation. In Proceedings of MICCAI , pages 424-432. Springer, 2016. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data
    § I. INTRODUCTION · p. 2

    The proposed global training objective in SR corresponds to a prior on the space of feasible high-resolution solutions and is experimentally useful because SR is an ill-posed problem.

    “The proposed global training objective in SR corresponds to a prior on the space of feasible high-resolution (HR) solutions, which is experimentally shown to be useful since SR is an illposed problem.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § Anatomically Constrained Neural Networks (ACNN): Application to Cardiac Image Enhancement and Segmentation · p. 1

    Priors are useful when images are corrupted or contain artefacts due to limitations in image acquisition.

    “In particular, priors can be useful in cases where images are corrupted and contain artefacts due to limitations in image acquisition.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § A. Clinical Motivation · p. 2

    The proposed method is useful not only for image enhancement and segmentation but also for studying anatomical shape variations in population studies and their associations with cardiac pathologies.

    “In that regard, the proposed method is not only useful for image enhancement and segmentation but also for the study of anatomical shape variations in population studies and their associations with cardiac related pathologies.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation
    § I. INTRODUCTION · p. 2

    The ACNN framework learns a non-linear compact representation of the underlying anatomy through a stacked convolutional autoencoder and enforces network predictions to follow the learnt statistical shape/label distributions.

    “Our framework learns a non-linear compact representation of the underlying anatomy through a stacked convolutional autoencoder [31] and enforces network predictions to follow the learnt statistical shape/label distributions.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [31] [31] J. Masci, U. Meier, D. Cires ¸an, and J. Schmidhuber. Stacked convolutional auto-encoders for hierarchical feature extraction. In International - Conference on Artificial Neural Networks , pages 52-59. Springer, 2011. -

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given
    § I. INTRODUCTION · p. 2

    The ACNN approach is independent of the particular NN architecture or application and can be combined with any state-of-the-art segmentation or super-resolution NN model.

    “More importantly, our approach is independent of the particular NN architecture or application; it can be combined with any of the state-of-the-art segmentation or super-resolution (SR) NN models and potentially improve its prediction accuracy and robustness without introducing any memory or computational complexity at inference time.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data
    § A. Clinical Motivation · p. 2

    The lower dimensional representations learnt by ACNN can be useful for classification of pathologies such as dilated and hypertrophic cardiomyopathy, and do not require point-wise correspondence search between subjects.

    “We also demonstrate that the lower dimensional representations learnt by the proposed ACNN can be useful for classification of pathologies such as dilated and hypertrophic cardiomyopathy, and it does not require point-wise correspondence search between subjects as in [39].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [39] [39] M. Shakeri, H. Lombaert, S. Tripathi, S. Kadoury, A. D. N. Initiative, et al. Deep spectral-based shape features for Alzheimers disease classification. In SASHIMI , pages 15-24. Springer, 2016. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Clinical Motivation · p. 2

    3D-US is recommended over 2D-US because it provides more accurate and reproducible volumetric measurements.

    “Similarly, 3D-US is recommended over the use of 2D-US since it has been demonstrated to provide more accurate and reproducible volumetric measurements [25].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [25] [25] R. M. Lang, L. P. Badano, W. Tsang, D. H. Adams, E. Agricola, T. Buck, F. F. Faletra, A. Franke, J. Hung, L. P. de Isla, et al. EAE/ASE recommendations for image acquisition and display using three-dimensional echocardiography. Journal of the American Society of Echocardiography , 25(1):3-46, 2012. -

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given
    § I. INTRODUCTION · p. 2

    ACNN can potentially improve prediction accuracy and robustness without introducing memory or computational complexity at inference time.

    “More importantly, our approach is independent of the particular NN architecture or application; it can be combined with any of the state-of-the-art segmentation or super-resolution (SR) NN models and potentially improve its prediction accuracy and robustness without introducing any memory or computational complexity at inference time.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § A. Clinical Motivation · p. 2

    Using the proposed approach, full 3D segmentation can be performed without explicit motion correction and without relying on LR slice-by-slice 2D segmentation.

    “Using the proposed approach we can perform full 3D segmentation without explicit motion correction and do not have to rely on LR slice-by-slice 2D segmentation.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation
    § E. Learnt Hidden Representations · p. 6

    Alain and Bengio showed that denoising reconstruction error is equivalent to a contractive penalty, which forces the encoder to resist input perturbations and contract input samples to similar low dimensional codes.

    “In [1], Alain and Bengio showed that the denoising reconstruction error is equivalent to contractive penalty, which forces the feature extraction (encoder) function f resist perturbations of the input and contracts these input samples to similar low dimensional codes.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [1] [1] G. Alain and Y. Bengio. What regularized auto-encoders learn from the data-generating distribution. Journal of Machine Learning Research , 15(1):3563-3593, 2014. -

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § C. Medical Image Super-Resolution (SR) with CNNs · p. 5

    Learnt shape representations can be used to regularise image SR, constraining the model to make only anatomically meaningful predictions.

    “As for the case of image segmentation, learnt shape representations can be used to regularise image SR, constraining the model to make only anatomically meaningful predictions.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § C. Medical Image Super-Resolution (SR) with CNNs · p. 5

    The proposed SR model generates image features in the LR image grid rather than early upsampling, which reduces memory and computation requirements significantly.

    “It provides two main advantages over the state-of-the-art medical image SR model proposed in [34]: (I) the network generates image features in the LR image grid rather than early upsampling of the features, which reduces memory and computation requirements significantly.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [34] [34] O. Oktay, W. Bai, M. Lee, R. Guerrero, K. Kamnitsas, J. Caballero, A. de Marvao, S. Cook, D. ORegan, and D. Rueckert. Multi-input cardiac image super-resolution using convolutional neural networks. In International Conference on MICCAI , pages 246-254. Springer, 2016. - source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation
    § E. Learnt Hidden Representations · p. 6

    The contractive penalty promotes the network to learn the underlying low-dimensional data manifold and capture its local smooth structure.

    “The given penalty function promotes the network to learn the underlying low-dimensional data manifold and capture its local smooth structure.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given
    § D. Cardiac 3D Ultrasound Image Segmentation · p. 9

    The 3D ultrasound experimental results were evaluated in a blinded fashion by uploading segmentations from 30 sequences to the CETUS web platform.

    “The experimental results were evaluated in a blinded fashion by uploading the generated segmentations from separate 30 sequences into the CETUS web platform.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § E. Learnt Hidden Representations · p. 6

    Low dimensional encoding enables training models with global characteristics and yields better generalisation power for the underlying anatomy.

    “Low dimensional encoding enables us to train models with global characteristics but also yields better generalisation power for the underlying anatomy as shown in earlier work [43].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [43] [43] A. Torralba and Y. Weiss. Small codes and large image databases for recognition. In Proceedings of CVPR , pages 1-8. IEEE, 2008. - source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § E. Learnt Hidden Representations · p. 6

    The extracted codes are expected to be correlated because the decoder merges some codes along the three spatial dimensions to construct input feature maps for transposed convolutions, but this is not a limitation.

    “In addition to the smoothness of the latent distributions, the extracted codes are expected to be correlated since the decoder merges some of the codes along the three spatial dimensions to construct input feature maps for the transposed convolutions, but this characteristic is not a limitation in our study.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisereasoning
    § C. Medical Image Super-Resolution (SR) with CNNs · p. 5

    Image SR is an ill-posed problem because there are infinite solutions for a given input sample but only a few are anatomically meaningful and accurate.

    “Image SR is an ill-posed problem as there are an infinite number of solutions for a given input sample but only a few would be anatomically meaningful and accurate.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § B. Convolutional Autoencoder Model and ACNN-Seg · p. 4

    The shape regularisation parametrisation encourages label consistency in model predictions and reduces false-positive detections.

    “In addition to imposing shape regularisation, this parametrisation encourages label consistency in model predictions, and reduces false-positive detections as they can influence the predicted codes in the hidden layer.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § D. Cardiac 3D Ultrasound Image Segmentation · p. 9

    Global regularisation reduces spurious false positive predictions and enforces abnormal LV segmentation shapes to fit the learnt representation model.

    “global regularisation reduces the amount of spurious false positive predictions and enforces abnormal LV segmentation shapes to fit into the learnt representation model.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § D. T-L Network Model and SR-ACNN · p. 5

    The T-L regulariser model is used only at training time and not during inference, so fully convolutional segmentation and SR models can still be used for applications using different image sizes.

    “It is important to note that the T-L regulariser model is used only at training time but not during inference; in other words, the fully convolutional (FCN) segmentation and super-resolution models can still be used for applications using different image sizes.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § IV. DISCUSSION AND CONCLUSION · p. 10

    Adversarial training comes at the expense of less interpretability of the regularisation term and unstable model training behaviour.

    “Moreover, we believe that adversarial training comes at the expense of less interpretability of the regularisation term and unstable model training behaviour, which still remains an open research problem.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § IV. DISCUSSION AND CONCLUSION · p. 10

    In super-resolution, adversarial loss enables the SR network to hallucinate fine texture detail and makes synthesized HR images appear qualitatively more realistic, but PSNR and SSIM scores are usually worse.

    “In SR application, AL enables the SR network to hallucinate fine texture detail, and the synthesized HR images appear qualitatively more realistic. However, at the same time the PSNR and SSIM scores are usually worse.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [26] [26] C. Ledig, L. Theis, F. Husz´ ar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, et al. Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint arXiv:1609.04802 , 2016. -

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § IV. DISCUSSION AND CONCLUSION · p. 10

    The proposed regulariser model can be seen as an application-specific training objective.

    “The proposed regulariser model can be seen as an application-specific training objective.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § E. Cardiac MR Image Enhancement · p. 10

    The 3D-Seg approach uses convolution kernels more efficiently without requiring the model to output a high-dimensional intensity image.

    “The 3DSeg approach uses the convolution kernels more efficiently without requiring the model to output a high-dimensional intensity image.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The clinical HR images used as reference for SR evaluation are accurate ground truth.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationnone given
    § IV. DISCUSSION AND CONCLUSION · p. 11

    The ACNN framework can be extended to other image analysis tasks where prior knowledge can provide model guidance and robustness.

    “The presented ACNN framework is not only limited to the medical image segmentation and SR tasks but can be extended to other image analysis tasks where prior knowledge can provide model guidance and robustness.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § IV. DISCUSSION AND CONCLUSION · p. 10

    Adversarial loss approaches have been used successfully in natural image super-resolution and segmentation tasks.

    “Such approaches have been used successfully in natural image super-resolution (SR) [26] and segmentation [30] tasks.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [26] [26] C. Ledig, L. Theis, F. Husz´ ar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, et al. Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint arXiv:1609.04802 , 2016. - source

      Not checked in the free run.

    • cites [30] [30] P. Luc, C. Couprie, S. Chintala, and J. Verbeek. Semantic segmentation using adversarial networks. In NIPS Workshop on Adversarial Training , 2016. - source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation
    § IV. DISCUSSION AND CONCLUSION · p. 10

    Adversarial training may not be suitable for medical applications, where accuracy and fidelity of visual content are more important than qualitative appearance of HR images.

    “For this reason, the authors of [26] have pointed out that adversarial training may not be suitable for medical applications, where the accuracy and fidelity of the visual content more important than the qualitative appearance of the HR images.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [26] [26] C. Ledig, L. Theis, F. Husz´ ar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, et al. Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint arXiv:1609.04802 , 2016. -

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § IV. DISCUSSION AND CONCLUSION · p. 11

    The additional K-L divergence term in a variational autoencoder would reduce the representation power of the AE, so local anatomical variations would not be captured in detail.

    “the additional K-L divergence term (constraint) would reduce the representation power of the AE; thus, the local anatomical variations would not be captured in detail.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § VI. SUPPLEMENTARY MATERIAL · p. 13

    Non-linear layers are not applied on lower-dimensional latent representations because that would further constrain the autoencoder.

    “Additionally, non-linear layers are not applied on the lower dimensional latent representations since that would further constrain the autoencoder.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § C. Cardiac Cine-MR Image Segmentation · p. 8

    Cascaded approaches add additional computational complexity due to increased filters, which could be redundant because a standard segmentation model can capture shape properties with a large receptive field and shape constraints.

    “More importantly, cascaded approaches add additional computational complexity due to the increased number of filters, which could be redundant given that the standard segmentation model is able to capture shape properties of the organs as long as it has a large receptive field and is optimised with shape constraints.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisenone given
    § IV. DISCUSSION AND CONCLUSION · p. 10

    Our AE model is trained solely on cardiac segmentation masks and its features are customised to identify anatomical variations observed in the heart chambers.

    “In contrast to this, our AE model is trained solely on cardiac segmentation masks and features are customised to identify anatomical variations observed in the heart chambers.”
    Quote in the paperSubstantial claim
46 of 46

  • The paper relies on this without support.

    The high-resolution motion-free cardiac MR images provide an appropriate reference for evaluating segmentation performance on low-resolution 2D stacks.

    “Since the ground-truth information is obtained from the HR motionfree images, the experimental results are expected to reflect the performance of the method with respect to an appropriate reference.”

  • The paper relies on this without support.

    A perfect mapping between intensity and label spaces is practically not achievable.

    “Since a perfect mapping between the intensity and label spaces is practically not achievable, the T-L learnt codes are expected to be more representative due to the inclusion of additional information.”

  • The paper relies on this without support.

    Anatomical objects in medical images are naturally more constrained in shape and location than objects in natural images.

    “Inclusion of priors in medical imaging could potentially have much more impact compared to their use in natural image analysis since anatomical objects in medical images are naturally more constrained in terms of their shape and location.”

  • The authors state this as an assumption.

    The proposed ACNN is one of the earliest studies demonstrating the use of convolutional autoencoder networks to learn anatomical shape variations from medical images.

    “More importantly, to the best of our knowledge, this is one of the earliest studies demonstrating the use of convolutional autoencoder networks to learn anatomical shape variations from medical images.”

  • The paper relies on this without support.

    The HR ground-truth label maps projected from HR image space are an accurate reference for segmentation evaluation.

    “The corresponding groundtruth label maps, however, are projected from the HR image space, which are annotated in the HR image grid.”

  • The paper relies on this without support.

    The clinical HR images used as reference for SR evaluation are accurate ground truth.

    “The quality of the upsampled images is evaluated in terms of SSIM metric [47] between the clinical HR image data and reconstructed HR images.”

  • The paper relies on this without support.

    The two expert cardiologists' MOS ratings reliably quantify image quality.

    “Two expert cardiologists were asked to rate the upsampled images from 1 (very poor) to 5 (excellent) based on the accuracy of the reconstructed LV boundary and geometry.”

  • The paper relies on this without support.

    The CETUS web platform evaluation is a valid blinded benchmark for segmentation accuracy.

    “The experimental results were evaluated in a blinded fashion by uploading the generated segmentations from separate 30 sequences into the CETUS web platform.”

  • The paper relies on this without support.

    The 2D stack LR images are a commonly used acquisition protocol for cardiac imaging.

    “only the 2D stack LR images are used, which is a commonly used acquisition protocol for cardiac imaging”

  • The paper relies on this without support.

    The LV blood pool is larger than the myocardium.

    “due to the larger size of the LV blood pool compared to the myocardium.”
10 of 10

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Duan et al. (2019) report performance superior to 3D-ACNN for myocardium segmentation, achieving 4.3% increase in Dice and 4.15mm (40% less) improvement in Hausdorff distance.

    “the authors report performance superior to 3D-ACNN Oktay et al. (2017), achieving 4.3% increase in Dice and 4.15mm (40% less) improvement in Hausdorff distance .”

    From Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation (2020) · cites this paper as Oktay et al. (2017)

  • Oktay et al. (2017) attribute the inferior performance by Ravishankar et al. (2017) to over-regularization, which they overcame by replacing CDAE with a frozen encoder during training.

    “The authors attribute the inferior performance by Ravishankar et al. (2017) to over-regularization, which they have overcome by replacing CDAE with a frozen encoder during training.”

    From Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation (2020) · cites this paper as Oktay et al. (2017)

  • Oktay et al. (2017) model achieves 1.2% and 2.0% improvement respectively in Dice over Ravishankar et al. (2017) for 3D endocardium and myocardium segmentation on a cine-MR dataset (1000 training and 200 testing).

    “The suggested model achieves 1.2% and 2.0% improvement respectively in Dice over Ravishankar et al. (2017) for 3D endocardium and myocardium segmentation on a cine-MR dataset (1000 training and 200 testing).”

    From Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation (2020) · cites this paper as Oktay et al. (2017)

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Cross-entropy and Euclidean gradients have their roles reversed

    Minor

    The text defines cross-entropy as the segmentation-reconstruction loss and Euclidean distance as the loss matching the predictor's latent codes. It then assigns the predictor-matching role to the first gradient, ∂L x, and the reconstruction role to the second, ∂L h-the reverse of those definitions.

1 of 1

Cites

Flagged first, then by citations

7 of the 60 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Oktay, O., Ferrante, E., Kamnitsas, K., Heinrich, M., Bai, W., Caballero, J., Cook, S. A., de Marvao, A., Dawes, T., O'Regan, D. P., Kainz, B., Glocker, B., & Rueckert, D.
Affiliations
Imperial College London (GB)
ORCID
Venue
IEEE Transactions on Medical Imaging
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2018-02
Volume
37
Issue
2
Pages
384-395
ISSN
  • 0278-0062 (ISSN-L)
  • 1558-254X
DOI
10.1109/tmi.2017.2743464
OpenAlex
W2620296437
Semantic Scholar
33f7b95903eeefdde37fabf9655b4de089d53bd2
Cited by
751
References
60
Citations by year
2026: 34 · 2025: 64 · 2024: 61 · 2023: 87 · 2022: 81 · 2021: 135 · 2020: 118 · 2019: 119 · 2018: 47 · 2017: 5
FWCI
32.79
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Medical Imaging (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • National University of Singapore (submittedVersion)
  • Spiral (Imperial College London) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
  • EPSRC Program (EP/P001009/1)
  • British Heart Foundation, U.K. (PG/12/27/29489)
  • National Institute for Health Research (NIHR) Biomedical Research Centre-based at Imperial College Healthcare NHS Trust
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
anatomical prior knowledge, cardiac image segmentation, biomarkers, artifact reduction
MeSH terms
Algorithms, Heart, Humans, Magnetic Resonance Imaging, Cardiomyopathies, Databases, Factual, Neural Networks, Computer, Imaging, Three-Dimensional, Cardiac Imaging Techniques

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review