DeepCut: Object Segmentation From Bounding Box Annotations Using Convolutional Neural Networks
Open accessAbstract
In this paper, we propose DeepCut, a method to obtain pixelwise object segmentations given an image dataset labelled weak annotations, in our case bounding boxes. It extends the approach of the well-known GrabCut [1] method to include machine learning by training a neural network classifier from bounding box annotations. We formulate the problem as an energy minimisation problem over a densely-connected conditional random field and iteratively update the training targets to obtain pixelwise object segmentations. Additionally, we propose variants of the DeepCut method and compare those to a naïve approach to CNN training under weak supervision. We test its applicability to solve brain and lung segmentation problems on a challenging fetal magnetic resonance dataset and obtain encouraging results in terms of accuracy.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Handling unseen objects is a major advantage over traditional CNNs and even transfer learning or weakly supervised learning, because for some objects it requires no annotated training instances.
“Dealing with unseen objects is a major advantage compared with traditional CNNs and even transfer learning [25] or weakly supervised learning [3], since for some objects it does not require annotated instances for training at all.”
From Interactive medical image segmentation using deep learning with image-specific fine tuning (2018) · cites this paper as [3]
These deep CNN interactive or semi-automatic methods are not fully interactive for testing since they do not accept further interactions for refinement.
“However, these methods are not fully interactive for testing since they do not accept further interactions for refinement.”
From DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation (2019) · cites this paper as [32]
DeepCut employs user-provided bounding boxes as annotations to train CNNs for the segmentation of fetal MRI.
“DeepCut [32] employs user-provided bounding boxes as annotations to train CNNs for the segmentation of fetal MRI.”
From DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation (2019) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Maximum Likelihood from Incomplete Data Via the <i>EM</i> Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444 · see citing works
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · see citing works
Fast approximate energy minimization via graph cuts
Boykov, Y., Veksler, O., & Zabih, R. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.969114 · Cited by 6,988 · see citing works
Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks
Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.222 · Cited by 3,150 · see citing works
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · see citing works
Automated fetal brain segmentation from 2D MRI slices for motion correction
Keraudren, K., Kuklisova-Murgasova, M., Kyriakopoulou, V., Malamateniou, C., Rutherford, M. A., Kainz, B., Hajnal, J. V., & Rueckert, D. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2014.07.023 · Cited by 91 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332 · see citing works
Interactive medical image segmentation using deep learning with image-specific fine tuning
Wang, G., Li, W., Zuluaga, M. A., Pratt, R., Patel, P. A., Aertsen, M., Doel, T., David, A. L., Deprest, J., Ourselin, S., et al. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2791721 · Cited by 843 · see citing works
A survey on active learning and human-in-the-loop deep learning for medical image analysis
Budd, S., Robinson, E. C., & Kainz, B. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102062 · Cited by 606 · see citing works
Self-citationDeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation
Wang, G., Zuluaga, M. A., Li, W., Pratt, R., Patel, P. A., Aertsen, M., Doel, T., David, A. L., Deprest, J., Ourselin, S., et al. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2840695 · Cited by 502 · see citing works
Structure Boundary Preserving Segmentation for Medical Image With Ambiguous Boundary
Lee, H. J., Kim, J. U., Lee, S., Kim, H. G., & Ro, Y. M. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00487 · Cited by 180 · see citing works
Luo, X., Wang, G., Song, T., Zhang, J., Aertsen, M., Deprest, J., Ourselin, S., Vercauteren, T., & Zhang, S. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102102 · Cited by 126 · see citing works
Machine Learning in Fetal Cardiology: What to Expect
Garcia-Canadilla, P., Sanchez-Martinez, S., Crispi, F., & Bijnens, B. (2020) · Fetal Diagnosis and Therapy · doi:10.1159/000505021 · Cited by 123 · see citing works
Volumetric memory network for interactive medical image segmentation
Zhou, T., Li, L., Bredell, G., Li, J., Unkelbach, J., & Konukoglu, E. (2023) · Medical Image Analysis · doi:10.1016/j.media.2022.102599 · Cited by 109 · see citing works
Segmentation and classification in MRI and US fetal imaging: Recent trends and future prospects
Torrents-Barrena, J., Piella, G., Masoller, N., Gratacós, E., Eixarch, E., Ceresa, M., & Ballester, M. Á. G. (2019) · Medical Image Analysis · doi:10.1016/j.media.2018.10.003 · Cited by 97 · see citing works
Weakly Supervised Estimation of Shadow Confidence Maps in Fetal Ultrasound Imaging
Meng, Q., Housden, J., Matthew, J., Rueckert, D., Schnabel, J. A., Kainz, B., Sinclair, M., Zimmer, V., Hou, B., Rajchl, M., et al. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2913311 · Cited by 52 · see citing works
Self-citationTorrents-Barrena, J., Monill, N., Piella, G., Gratacós, E., Eixarch, E., Ceresa, M., & González Ballester, M. A. (2021) · Academic Radiology · doi:10.1016/j.acra.2019.11.006 · Cited by 42 · see citing works
Tian, Z., He, Y., Sun, L., Li, Y., & Du, S. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128184 · Cited by 5 · see citing works
Qian, C., Yang, H., & Li, J. (2023) · 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC) · doi:10.1109/smc53992.2023.10393973 · Cited by 1 · see citing works
- Authors
- Rajchl, M., Lee, M. C. H., Oktay, O., Kamnitsas, K., Passerat-Palmbach, J., Bai, W., Damodaram, M., Rutherford, M. A., Hajnal, J. V., Kainz, B., & Rueckert, D.
- Affiliations
- Imperial College London (GB)
- Department of Computing, Imperial College London, SW7 2AZ, London, U.K
- Hammersmith Hospital (GB)
- King's College London (GB)
- ORCID
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2017-02
- Volume
- 36
- Issue
- 2
- Pages
- 674-683
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- PMC
- PMC7115996
- OpenAlex
- W2396622801
- Cited by
- 410
- References
- 7
- Citations by year
- 2026: 21 · 2025: 35 · 2024: 37 · 2023: 60 · 2022: 61 · 2021: 62 · 2020: 54 · 2019: 45 · 2018: 22 · 2017: 11 · 2016: 2
- FWCI
- 18.14
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- Research Portal (King's College London) (submittedVersion)
- Spiral (Imperial College London) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Wellcome Trust and EPSRC IEH (102431)
- iFIND project
- Developing Human Connectome Project
- Synergy Grant by the European Research Council
- European Union’s Seventh Framework Programme (FP/2007-2013)
- ERC (319456)
- National Institute for Health Research (NIHR) Biomedical Research Centre based at Guy’s and St Thomas’ NHS Foundation Trust and King’s College London
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- object segmentation, bounding box annotation, weakly supervised learning, GrabCut, conditional random fields, energy minimization, convolutional neural networks, brain segmentation, lung segmentation, fetal MRI
- MeSH terms
- Machine Learning, Algorithms, Brain, Humans, Image Enhancement, Image Interpretation, Computer-Assisted, Magnetic Resonance Imaging, Monte Carlo Method, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
