Automatic 3d bi-ventricular segmentation of cardiac images by a shape-refined multi-task deep learning approach
Open accessAbstract
Deep learning approaches have achieved state-of-the-art performance in cardiac magnetic resonance (CMR) image segmentation. However, most approaches have focused on learning image intensity features for segmentation, whereas the incorporation of anatomical shape priors has received less attention. In this paper, we combine a multi-task deep learning approach with atlas propagation to develop a shape-refined bi-ventricular segmentation pipeline for short-axis CMR volumetric images. The pipeline first employs a fully convolutional network (FCN) that learns segmentation and landmark localization tasks simultaneously. The architecture of the proposed FCN uses a 2.5D representation, thus combining the computational advantage of 2D FCNs networks and the capability of addressing 3D spatial consistency without compromising segmentation accuracy. Moreover, a refinement step is designed to explicitly impose shape prior knowledge and improve segmentation quality. This step is effective for overcoming image artifacts (e.g., due to different breath-hold positions and large slice thickness), which preclude the creation of anatomically meaningful 3D cardiac shapes. The pipeline is fully automated, due to network's ability to infer landmarks, which are then used downstream in the pipeline to initialize atlas propagation. We validate the pipeline on 1831 healthy subjects and 649 subjects with pulmonary hypertension. Extensive numerical experiments on the two datasets demonstrate that our proposed method is robust and capable of producing accurate, high-resolution, and anatomically smooth bi-ventricular 3D models, despite the presence of artifacts in input CMR volumes.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Duan et al. (2019) refinement stage is computationally expensive, taking 15-20 minutes per case with their multiple CPU implementation.
“However, the refinement stage is computationally expensive, taking 15-20 minutes per case with their multiple CPU implementation.”
From Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation (2020) · cites this paper as Duan et al. (2019)
Duan et al. (2019) 2.5D FCN trained on LR cases with refinement achieves comparable performance relative to the 2.5D FCN trained on HR cases, trailing by 2.0% for most regions in Dice and 0.5mm worse in Hausdorff distance.
“the 2.5D FCN trained on the LR cases with refinement achieves comparable performance relative to the 2.5D FCN trained on the HR cases, trailing by 2.0% for most regions in Dice and 0.5mm worse in Hausdorff distance.”
From Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation (2020) · cites this paper as Duan et al. (2019)
Duan et al. (2019) report performance superior to 3D-ACNN for myocardium segmentation, achieving 4.3% increase in Dice and 4.15mm (40% less) improvement in Hausdorff distance.
“the authors report performance superior to 3D-ACNN Oktay et al. (2017), achieving 4.3% increase in Dice and 4.15mm (40% less) improvement in Hausdorff distance .”
From Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation (2020) · cites this paper as Duan et al. (2019)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation
Milletari, F., Navab, N., & Ahmadi, S.-A. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.79 · Cited by 11,931 · see citing works
3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
Çiçek, Ö., Abdulkadir, A., Lienkamp, S. S., Brox, T., & Ronneberger, O. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_49 · Cited by 7,945 · see citing works
A Non-Local Algorithm for Image Denoising
Buades, A., Coll, B., & Morel, J.-M. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.38 · Cited by 7,616 · see citing works
Nonrigid registration using free-form deformations: application to breast mr images
Rueckert, D., Sonoda, L. I., Hayes, C., Hill, D. L. G., Leach, M. O., & Hawkes, D. J. (1999) · IEEE Transactions on Medical Imaging · doi:10.1109/42.796284 · Cited by 5,265 · see citing works
Self-citationEfficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation
Kamnitsas, K., Ledig, C., Newcombe, V. F. J., Simpson, J. P., Kane, A. D., Menon, D. K., Rueckert, D., & Glocker, B. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.10.004 · Cited by 3,498 · cited 1× · see citing works
“Workarounds such as subsampling [25] or use of small batch size and fewer convolutional layers [5], [7], [11] are often considered when training 3D networks, but these either complicate the”
BackgroundSelf-citationBernard, O., Lalande, A., Zotti, C., Cervenansky, F., Yang, X., Heng, P.-A., Cetin, I., Lekadir, K., Camara, O., Gonzalez Ballester, M. A., et al. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2837502 · Cited by 2,431 · see citing works
Automated cardiovascular magnetic resonance image analysis with fully convolutional networks
Bai, W., Sinclair, M., Tarroni, G., Oktay, O., Rajchl, M., Vaillant, G., Lee, A. M., Aung, N., Lukaschuk, E., Sanghvi, M. M., et al. (2018) · Journal of Cardiovascular Magnetic Resonance · doi:10.1186/s12968-018-0471-x · Cited by 773 · see citing works
Self-citationOktay, O., Ferrante, E., Kamnitsas, K., Heinrich, M., Bai, W., Caballero, J., Cook, S. A., de Marvao, A., Dawes, T., O'Regan, D. P., et al. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2743464 · Cited by 751 · see citing works
Self-citationA review of segmentation methods in short axis cardiac MR images
Petitjean, C., & Dacher, J. (2010) · Medical Image Analysis · doi:10.1016/j.media.2010.12.004 · Cited by 732 · see citing works
Avendi, M. R., Kheradvar, A., & Jafarkhani, H. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.01.005 · Cited by 591 · see citing works
Deep hashing for compact binary codes learning
Liong, V. E., Lu, J., Wang, G., Moulin, P., & Zhou, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298862 · Cited by 545 · see citing works
UK Biobank's cardiovascular magnetic resonance protocol
Petersen, S. E., Matthews, P. M., Francis, J. M., Robson, M. D., Zemrak, F., Boubertakh, R., Young, A. A., Hudson, S., Weale, P., Garratt, S., et al. (2016) · Journal of Cardiovascular Magnetic Resonance · doi:10.1186/s12968-016-0227-4 · Cited by 481 · see citing works
Shape-based interpolation of multidimensional objects
Raya, S. P., & Udupa, J. K. (1990) · IEEE Transactions on Medical Imaging · doi:10.1109/42.52980 · Cited by 440 · see citing works
Frangi, A. F., Rueckert, D., Schnabel, J. A., & Niessen, W. J. (2002) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2002.804426 · Cited by 382 · see citing works
Self-citationKhened, M., Kollerathu, V. A., & Krishnamurthi, G. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.10.004 · Cited by 376 · see citing works
Ngo, T. A., Lu, Z., & Carneiro, G. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.05.009 · Cited by 334 · see citing works
A Fully Convolutional Neural Network for Cardiac Segmentation in Short-Axis MRI
Tran, P. V. (2016) · arXiv · doi:10.48550/arxiv.1604.00494 · Cited by 323 · see citing works
Isensee, F., Jaeger, P. F., Full, P. M., Ivo Wolf, A., Engelhardt, S., & Maier-Hein, K. H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-75541-0_13 · Cited by 313 · see citing works
Multi-input Cardiac Image Super-Resolution Using Convolutional Neural Networks
Oktay, O., Bai, W., Lee, M., Guerrero, R., Kamnitsas, K., Caballero, J., de Marvao, A., Cook, S., O’Regan, D., & Rueckert, D. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46726-9_29 · Cited by 246 · see citing works
Self-citationAn Exploration of 2D and 3D Deep Learning Techniques for Cardiac MR Image Segmentation
Baumgartner, C. F., Koch, L. M., Pollefeys, M., & Ender Konukoglu, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-75541-0_12 · Cited by 234 · see citing works
Bai, W., Shi, W., O’Regan, D. P., Tong, T., Wang, H., Jamil‐Copley, S., Peters, N. S., & Rueckert, D. (2013) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2256922 · Cited by 209 · see citing works
Self-citation3-D Consistent and Robust Segmentation of Cardiac Images by Deep Learning With Spatial Propagation
Zheng, Q., Delingette, H., Duchateau, N., & Ayache, N. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2820742 · Cited by 202 · see citing works
Multi-atlas segmentation with augmented features for cardiac MR images
Bai, W., Shi, W., Ledig, C., & Rueckert, D. (2014) · Medical Image Analysis · doi:10.1016/j.media.2014.09.005 · Cited by 159 · see citing works
Self-citationLearning and Incorporating Shape Models for Semantic Segmentation
Ravishankar, H., Venkataramani, R., Thiruvenkadam, S. R., Sudhakar, P., & Vaidya, V. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66182-7_24 · Cited by 155 · see citing works
2D-3D Fully Convolutional Neural Networks for Cardiac MR Segmentation
Patravali, J., Jain, S., & Chilamkurthy, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-75541-0_14 · Cited by 130 · see citing works
CardiacNET: Segmentation of Left Atrium and Proximal Pulmonary Veins from MRI Using Multi-view CNN
Mortazi, A., Karim, R., Rhode, K., et al. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66185-8_43 · Cited by 101 · see citing works
Statistical Atlases and Computational Models of the Heart. ACDC and MMWHS Challenges
Isensee, F., Jaeger, P. F., Full, P. M., Ivo Wolf, A., Engelhardt, S., & Maier-Hein, K. H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-75541-0 · Cited by 86 · see citing works
Complete valvular heart apparatus model from 4D cardiac CT
Grbić, S., Ionasec, R. I., Vitanovski, D., Voigt, I., Wang, Y., Georgescu, B., Navab, N., & Comaniciu, D. (2012) · Medical Image Analysis · doi:10.1016/j.media.2012.02.003 · Cited by 66 · see citing works
Learning-Based Quality Control for Cardiac MR Images
Tarroni, G., Oktay, O., Bai, W., Schuh, A., Suzuki, H., Passerat‐Palmbach, J., de Marvao, A., O’Regan, D. P., Cook, S. A., Glocker, B., et al. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2878509 · Cited by 66 · see citing works
Self-citationCardiovascular magnetic resonance imaging: what the general cardiologist should know
Ripley, D. P., Musa, T. A., Dobson, L. E., Plein, S., & Greenwood, J. P. (2016) · Heart · doi:10.1136/heartjnl-2015-307896 · Cited by 57 · see citing works
Winther, H. B., Hundt, C., Schmidt, B., et al. (2018) · JACC. Cardiovascular imaging · doi:10.1016/j.jcmg.2017.11.013 · Cited by 56 · see citing works
Left Ventricle Segmentation in Cardiac MR Images Using Fully Convolutional Network
Nasr-Esfahani, M., Mohrekesh, M., Akbari, M., et al. (2018) · doi:10.1109/embc.2018.8512536 · Cited by 53 · see citing works
Fast algorithm for color texture image inpainting using the non-local CTV model
Duan, J., Pan, Z., Zhang, B., et al. (2015) · Journal of Global Optimization · doi:10.1007/s10898-015-0290-7 · Cited by 48 · see citing works
Self-citationde Marvao, A., Dawes, T. J. W., Shi, W., Minas, C., Keenan, N., Diamond, T., Durighel, G., Montana, G., Rueckert, D., Cook, S. A., et al. (2014) · Journal of Cardiovascular Magnetic Resonance · doi:10.1186/1532-429x-16-16 · Cited by 48 · see citing works
Self-citationCardiac MR Segmentation from Undersampled k-space Using Deep Latent Representation Learning
Schlemper, J., Oktay, O., Bai, W., Castro, D. C., Duan, J., Qin, C., Hajnal, J. V., & Rueckert, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00928-1_30 · Cited by 43 · see citing works
Self-citationDensely-Connected Multi-Magnification Hashing for Histopathological Image Retrieval
Gu, Y., & Yang, J. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2018.2882647 · Cited by 35 · see citing works
Learning clinically useful information from images: Past, present and future
Rueckert, D., Glocker, B., & Kainz, B. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.06.009 · Cited by 33 · see citing works
Self-citationLu, W., Duan, J., Orive-Miguel, D., et al. (2019) · Biomedical Optics Express · doi:10.1364/boe.10.002684 · Cited by 32 · see citing works
Self-citationDuan, J., Schlemper, J., Bai, W., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00937-3_68 · Cited by 31 · see citing works
Self-citationChallenges of Cardiac Image Analysis in Large-Scale Population-Based Studies
Medrano−Gracia, P., Cowan, B. R., Suinesiaputra, A., & Young, A. A. (2015) · Current Cardiology Reports · doi:10.1007/s11886-015-0563-2 · Cited by 19 · see citing works
$\nu$-net: Deep Learning for Generalized Biventricular Cardiac Mass and Function ParametersPreprint
Winther, H. B., Hundt, C., Schmidt, B., et al. (2017) · arXiv (Cornell University) · Cited by 14 · see citing works
Duan, J., Schlemper, J., Bai, W., Dawes, T. J. W., Bello, G., Biffi, C., Doumou, G., de Marvao, A., O’Regan, D. P., & Rueckert, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-04747-4_24 · Cited by 3 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- [33]Duan J Pan Z Liu W Tai X-C Color texture image inpainting using the non local ctv model Journal of Signal and Information Processing 2013 4 03 43
Cited by
Sorted by citations
Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation
Tajbakhsh, N., Jeyaseelan, L., Li, Q., Chiang, J. N., Wu, Z., & Ding, X. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101693 · Cited by 1,095 · see citing works
“More recently, Duan et al. (2019) propose a refinement approach following a segmentation network that utilizes shape”
BackgroundXu, L., Liu, M., Shen, Z., Wang, H., Liu, X., Wang, X., Wang, S., Li, T., Yu, S., Hou, M., et al. (2020) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2019.101690 · Cited by 87 · see citing works
- Authors
- Duan, J., Bello, G., Schlemper, J., Bai, W., Dawes, T. J. W., Biffi, C., de Marvao, A., Doumoud, G., O'Regan, D. P., & Rueckert, D.
- Affiliations
- MRC London Institute of Medical Sciences (GB)
- Imperial College London (GB)
- ORCID
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2019-09
- Volume
- 38
- Issue
- 9
- Pages
- 2151-2164
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- PMC
- PMC6728160
- OpenAlex
- W2888570268
- Cited by
- 227
- References
- 42
- Citations by year
- 2026: 21 · 2025: 18 · 2024: 25 · 2023: 43 · 2022: 34 · 2021: 39 · 2020: 35 · 2019: 12
- FWCI
- 13.27
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion, cc-by)
- PubMed (publishedVersion)
- University of Birmingham Research Portal (University of Birmingham) (publishedVersion, cc-by)
- PubMed Central (acceptedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Spiral (Imperial College London) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- British Heart Foundation (NH/17/1/32725, RE/13/4/30184)
- Engineering and Physical Sciences Research Council (EP/P001009/1)
- National Institute for Health Research
- Medical Research Council
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Cardiac Imaging and Diagnostics
- Physical Sciences › Engineering › Biomedical Engineering › Medical Imaging and Analysis
- Keywords
- cardiac image segmentation, cardiac magnetic resonance, multi-task learning, landmark detection, shape priors, pulmonary hypertension, image artifacts, 3D heart model
- MeSH terms
- Deep Learning, Algorithms, Heart, Humans, Magnetic Resonance Imaging, Cine, Imaging, Three-Dimensional, Cardiac Imaging Techniques
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
