Anatomically Constrained Neural Networks (ACNNs): Application to Cardiac Image Enhancement and Segmentation

In the paper

“We show that the proposed segmentation and SR models become more robust against imaging artefacts mentioned earlier which is underlined by our state-of-the-art results on the MICCAI'14 CETUS public benchmark [8].”

What the sources say

• Not checked yetcites [8] [8] O. Bernard, J. G. Bosch, B. Heyde, M. Alessandrini, D. Barbosa, S. Camarasu-Pop, F. Cervenansky, S. Valette, O. Mirea, M. Bernier, et al. Standardized evaluation system for left ventricular segmentation algorithms in 3D echocardiography. IEEE TMI , 35(4):967-977, 2016. - source

In the paper

“(III) An image SR technique that extends previous work [34] and that is robust against slice misalignments;”

What the sources say

• Not checked yetcites [34] [34] O. Oktay, W. Bai, M. Lee, R. Guerrero, K. Kamnitsas, J. Caballero, A. de Marvao, S. Cook, D. ORegan, and D. Rueckert. Multi-input cardiac image super-resolution using convolutional neural networks. In International Conference on MICCAI , pages 246-254. Springer, 2016. - source

In the paper

“our approach is computationally more efficient than the state-of-the-art SR-CNN model [34] as the feature extraction is performed in the low-dimensional image space.”

What the sources say

• Not checked yetcites [34] [34] O. Oktay, W. Bai, M. Lee, R. Guerrero, K. Kamnitsas, J. Caballero, A. de Marvao, S. Cook, D. ORegan, and D. Rueckert. Multi-input cardiac image super-resolution using convolutional neural networks. In International Conference on MICCAI , pages 246-254. Springer, 2016. - source

In the paper

“ACNN-Seg | 1.37 ± 0.42 | 7.89 ± 3.83 | .939 ± .017 | 1.14 ± 0.22 | 7.31 ± 3.59 | .811 ± .027 | 1 . 60 × 10 6 |”

In the paper

“It is observed that the learnt latent representations in general follow a normal distribution and they are not separated in multi-clusters ( e.g. mixture of Gaussians).”

In the paper

“In particular, Hausdorff distances were reduced significantly”

In the paper

“Previous slice by slice segmentation approaches validated their methods on LR annotations; however, we see that the produced label maps are far off from the true underlying ventricular geometry and it can be a limiting factor for the analysis of ventricle morphology.”

In the paper

“Similarly, we observed an improvement in terms of normalised Dice score, which was quantitatively not significant due to large volumetric size of the LV cavity.”

In the paper

“It is observed that the ACNN approach ensures better consistency between frames although none of the methods have used temporal information.”

In the paper

“We also analysed the performance change in AE-Seg with respect to the number of parameters, which shows that the small capacity AE-Seg model ( 8 × 10 4 pars) is not suitable for cardiac image segmentation as the second stage in the cascaded model does not improve the performance significantly.”

In the paper

“We observed that reconstruction of label-maps from low dimensional representations was limited since the ventricle boundaries were not delineated properly but rather a rough segmentation was generated (DSC: . 734 ).”

In the paper

“In the segmentation problem, the Sorensen-Dice loss was tested in the experiments as an alternative to the cross-entropy loss, yet we observed a degraded performance since the latter is a smoother function.”

In the paper

“By including the codes from ES phase, the classification accuracies were improved to 86 . 6% vs 91 . 6% .”

In the paper

“We used 10 -fold cross-validation on 60 CMR sequences and obtained 76 . 6% vs 83 . 3% accuracy using PCA and T-L codes extracted from ED phase.”

In the paper

“It is also observed to be more memory efficient compared to the standard 3D-UNet architecture [12].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [11] [11] H. Chen, Y. Zheng, J.-H. Park, P.-A. Heng, and S. K. Zhou. Iterative multi-domain regularized deep learning for anatomical structure detection and segmentation from ultrasound images. In International Conference on MICCAI , pages 487-495. Springer, 2016. -
• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [12] [12] ¨ O. C ¸ ic ¸ek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger. 3D U-Net: Learning dense volumetric segmentation from sparse annotation. In Proceedings of MICCAI , pages 424-432. Springer, 2016. -

In the paper

“The proposed global training objective in SR corresponds to a prior on the space of feasible high-resolution (HR) solutions, which is experimentally shown to be useful since SR is an illposed problem.”

In the paper

“In particular, priors can be useful in cases where images are corrupted and contain artefacts due to limitations in image acquisition.”

In the paper

“In that regard, the proposed method is not only useful for image enhancement and segmentation but also for the study of anatomical shape variations in population studies and their associations with cardiac related pathologies.”

In the paper

“Our framework learns a non-linear compact representation of the underlying anatomy through a stacked convolutional autoencoder [31] and enforces network predictions to follow the learnt statistical shape/label distributions.”

What the sources say

• Not checked yetcites [31] [31] J. Masci, U. Meier, D. Cires ¸an, and J. Schmidhuber. Stacked convolutional auto-encoders for hierarchical feature extraction. In International - Conference on Artificial Neural Networks , pages 52-59. Springer, 2011. -

In the paper

“More importantly, our approach is independent of the particular NN architecture or application; it can be combined with any of the state-of-the-art segmentation or super-resolution (SR) NN models and potentially improve its prediction accuracy and robustness without introducing any memory or computational complexity at inference time.”

In the paper

“We also demonstrate that the lower dimensional representations learnt by the proposed ACNN can be useful for classification of pathologies such as dilated and hypertrophic cardiomyopathy, and it does not require point-wise correspondence search between subjects as in [39].”

What the sources say

• ?Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  cites [39] [39] M. Shakeri, H. Lombaert, S. Tripathi, S. Kadoury, A. D. N. Initiative, et al. Deep spectral-based shape features for Alzheimers disease classification. In SASHIMI , pages 15-24. Springer, 2016. -

In the paper

“Similarly, 3D-US is recommended over the use of 2D-US since it has been demonstrated to provide more accurate and reproducible volumetric measurements [25].”

What the sources say

• Not checked yetcites [25] [25] R. M. Lang, L. P. Badano, W. Tsang, D. H. Adams, E. Agricola, T. Buck, F. F. Faletra, A. Franke, J. Hung, L. P. de Isla, et al. EAE/ASE recommendations for image acquisition and display using three-dimensional echocardiography. Journal of the American Society of Echocardiography , 25(1):3-46, 2012. -

In the paper

“More importantly, our approach is independent of the particular NN architecture or application; it can be combined with any of the state-of-the-art segmentation or super-resolution (SR) NN models and potentially improve its prediction accuracy and robustness without introducing any memory or computational complexity at inference time.”

In the paper

“Using the proposed approach we can perform full 3D segmentation without explicit motion correction and do not have to rely on LR slice-by-slice 2D segmentation.”

In the paper

“In [1], Alain and Bengio showed that the denoising reconstruction error is equivalent to contractive penalty, which forces the feature extraction (encoder) function f resist perturbations of the input and contracts these input samples to similar low dimensional codes.”

What the sources say

• Not checked yetcites [1] [1] G. Alain and Y. Bengio. What regularized auto-encoders learn from the data-generating distribution. Journal of Machine Learning Research , 15(1):3563-3593, 2014. -

In the paper

“As for the case of image segmentation, learnt shape representations can be used to regularise image SR, constraining the model to make only anatomically meaningful predictions.”

In the paper

“It provides two main advantages over the state-of-the-art medical image SR model proposed in [34]: (I) the network generates image features in the LR image grid rather than early upsampling of the features, which reduces memory and computation requirements significantly.”

What the sources say

• Not checked yetcites [34] [34] O. Oktay, W. Bai, M. Lee, R. Guerrero, K. Kamnitsas, J. Caballero, A. de Marvao, S. Cook, D. ORegan, and D. Rueckert. Multi-input cardiac image super-resolution using convolutional neural networks. In International Conference on MICCAI , pages 246-254. Springer, 2016. - source

In the paper

“The given penalty function promotes the network to learn the underlying low-dimensional data manifold and capture its local smooth structure.”

In the paper

“The experimental results were evaluated in a blinded fashion by uploading the generated segmentations from separate 30 sequences into the CETUS web platform.”

In the paper

“Low dimensional encoding enables us to train models with global characteristics but also yields better generalisation power for the underlying anatomy as shown in earlier work [43].”

What the sources say

• Not checked yetcites [43] [43] A. Torralba and Y. Weiss. Small codes and large image databases for recognition. In Proceedings of CVPR , pages 1-8. IEEE, 2008. - source

In the paper

“In addition to the smoothness of the latent distributions, the extracted codes are expected to be correlated since the decoder merges some of the codes along the three spatial dimensions to construct input feature maps for the transposed convolutions, but this characteristic is not a limitation in our study.”

In the paper

“Image SR is an ill-posed problem as there are an infinite number of solutions for a given input sample but only a few would be anatomically meaningful and accurate.”

In the paper

“In addition to imposing shape regularisation, this parametrisation encourages label consistency in model predictions, and reduces false-positive detections as they can influence the predicted codes in the hidden layer.”

In the paper

“global regularisation reduces the amount of spurious false positive predictions and enforces abnormal LV segmentation shapes to fit into the learnt representation model.”

In the paper

“It is important to note that the T-L regulariser model is used only at training time but not during inference; in other words, the fully convolutional (FCN) segmentation and super-resolution models can still be used for applications using different image sizes.”

In the paper

“Moreover, we believe that adversarial training comes at the expense of less interpretability of the regularisation term and unstable model training behaviour, which still remains an open research problem.”

In the paper

“In SR application, AL enables the SR network to hallucinate fine texture detail, and the synthesized HR images appear qualitatively more realistic. However, at the same time the PSNR and SSIM scores are usually worse.”

What the sources say

• Not checked yetcites [26] [26] C. Ledig, L. Theis, F. Husz´ ar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, et al. Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint arXiv:1609.04802 , 2016. -

In the paper

“The proposed regulariser model can be seen as an application-specific training objective.”

In the paper

“The 3DSeg approach uses the convolution kernels more efficiently without requiring the model to output a high-dimensional intensity image.”

In the paper

“The presented ACNN framework is not only limited to the medical image segmentation and SR tasks but can be extended to other image analysis tasks where prior knowledge can provide model guidance and robustness.”

In the paper

“Such approaches have been used successfully in natural image super-resolution (SR) [26] and segmentation [30] tasks.”

What the sources say

• Not checked yetcites [26] [26] C. Ledig, L. Theis, F. Husz´ ar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, et al. Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint arXiv:1609.04802 , 2016. - source
• Not checked yetcites [30] [30] P. Luc, C. Couprie, S. Chintala, and J. Verbeek. Semantic segmentation using adversarial networks. In NIPS Workshop on Adversarial Training , 2016. - source

In the paper

“For this reason, the authors of [26] have pointed out that adversarial training may not be suitable for medical applications, where the accuracy and fidelity of the visual content more important than the qualitative appearance of the HR images.”

What the sources say

• Not checked yetcites [26] [26] C. Ledig, L. Theis, F. Husz´ ar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, et al. Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint arXiv:1609.04802 , 2016. -

In the paper

“the additional K-L divergence term (constraint) would reduce the representation power of the AE; thus, the local anatomical variations would not be captured in detail.”

In the paper

“Additionally, non-linear layers are not applied on the lower dimensional latent representations since that would further constrain the autoencoder.”

In the paper

“More importantly, cascaded approaches add additional computational complexity due to the increased number of filters, which could be redundant given that the standard segmentation model is able to capture shape properties of the organs as long as it has a large receptive field and is optimised with shape constraints.”

In the paper

“In contrast to this, our AE model is trained solely on cardiac segmentation masks and features are customised to identify anatomical variations observed in the heart chambers.”