Deep Interactive Object Selection
Open accessAbstract
Interactive object selection is a very important research problem and has many applications. Previous algorithms require substantial user interactions to estimate the foreground and background distributions. In this paper, we present a novel deep-learning-based algorithm which has much better understanding of objectness and can reduce user interactions to just a few clicks. Our algorithm transforms user-provided positive and negative clicks into two Euclidean distance maps which are then concatenated with the RGB channels of images to compose (image, user interactions) pairs. We generate many of such pairs by combining several random sampling strategies to model users' click patterns and use them to finetune deep Fully Convolutional Networks (FCNs). Finally the output probability maps of our FCN-8s model is integrated with graph cut optimization to refine the boundary segments. Our model is trained on the PASCAL segmentation dataset and evaluated on other datasets with different object classes. Experimental results on both seen and unseen objects demonstrate that our algorithm has a good generalization ability and is superior to all existing interactive object selection approaches.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In [40], a deep interactive object selection method transforms user-provided clicks into euclidean distance maps and concatenates them with the input of FCNs.
“In [40], a deep interactive object selection method was proposed where user-provided clicks are transformed into euclidean distance maps and then concatenated with the input of FCNs.”
From DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation (2019) · cites this paper as [40]
Cites
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · see citing works
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · see citing works
Kass, M., Witkin, A., & Terzopoulos, D. (1988) · International Journal of Computer Vision · doi:10.1007/bf00133570 · Cited by 17,165 · see citing works
Normalized cuts and image segmentation
Shi, J., & Malik, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.868688 · Cited by 15,842 · see citing works
GrabCut: Interactive foreground extraction using iterated graph cuts
Rother, C., Kolmogorov, V., & Blake, A. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015720 · Cited by 5,703 · see citing works
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213 · see citing works
Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images
Boykov, Y. Y., & Jolly, M.-P. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937505 · Cited by 3,674 · see citing works
Random Walks for Image Segmentation
Grady, L. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.233 · Cited by 2,624 · see citing works
Learning Hierarchical Features for Scene Labeling
Farabet, C., Couprie, C., Najman, L., & LeCun, Y. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.231 · Cited by 2,620 · see citing works
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · see citing works
Li, Y., Sun, J. Z., Tang, C., & Shum, H. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015719 · Cited by 1,097 · see citing works
Semantic texton forests for image categorization and segmentation
Shotton, J., Johnson, M., & Cipolla, R. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587503 · Cited by 1,027 · see citing works
Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation
Lin, G., Shen, C., van den Hengel, A., & Reid, I. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.348 · Cited by 862 · see citing works
Intelligent scissors for image composition
Mortensen, E. N., & Barrett, W. A. (1995) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/218380.218442 · Cited by 789 · see citing works
Semantic Image Segmentation via Deep Parsing Network
Liu, Z., Li, X., Luo, P., Loy, C.-C., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.162 · Cited by 675 · see citing works
SuperParsing: Scalable Nonparametric Image Parsing with Superpixels
Tighe, J., & Lazebnik, S. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15555-0_26 · Cited by 481 · see citing works
Parsing clothing in fashion photographs
Yamaguchi, K., Kiapour, M. H., Ortiz, L. E., & Berg, T. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248101 · Cited by 476 · see citing works
Geodesic star convexity for interactive image segmentation
Gulshan, V., Rother, C., Criminisi, A., et al. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540073 · Cited by 414 · see citing works
A comparative evaluation of interactive segmentation algorithms
McGuinness, K., & O'Connor, N. E. (2009) · Pattern Recognition · doi:10.1016/j.patcog.2009.03.008 · Cited by 313 · see citing works
Geodesic Matting: A Framework for Fast Interactive Image and Video Segmentation and Matting
Xue, B., & Sapiro, G. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-008-0191-z · Cited by 300 · see citing works
GeoS: Geodesic Image Segmentation
Criminisi, A., Sharp, T., & Blake, A. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-88682-2_9 · Cited by 277 · see citing works
Li, Y., Sun, J. Z., Tang, C., et al. (2004) · ACM SIGGRAPH Conference Papers (SIGGRAPH) · doi:10.1145/1186562.1015719 · Cited by 249 · see citing works
Finding Things: Image Parsing with Regions and Per-Exemplar Detectors
Tighe, J., & Lazebnik, S. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.386 · Cited by 207 · see citing works
Geodesic graph cut for interactive image segmentation
Price, B. L., Morse, B. S., & Cohen, S. D. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540079 · Cited by 186 · see citing works
Self-citationContext Driven Scene Parsing with Attention to Rare Classes
Yang, J., Price, B. L., Cohen, S. D., & Yang, M. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.415 · Cited by 148 · see citing works
Self-citationWeakly-Supervised Dual Clustering for Image Semantic Segmentation
Liu, Y., Liu, J., Li, Z., et al. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.270 · Cited by 69 · see citing works
Lin, W., Zhang, Y., Lu, J., Zhou, B., Wang, J., & Zhou, Y. (2014) · Neurocomputing · doi:10.1016/j.neucom.2014.12.044 · Cited by 57 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00371 · Cited by 11,953 · see citing works
“Our default point sampling follows standard practice in interactive segmentation [109, 64, 92].”
BackgroundMethodInteractive medical image segmentation using deep learning with image-specific fine tuning
Wang, G., Li, W., Zuluaga, M. A., Pratt, R., Patel, P. A., Aertsen, M., Doel, T., David, A. L., Deprest, J., Ourselin, S., et al. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2791721 · Cited by 843 · see citing works
A survey of human-in-the-loop for machine learning
Wu, X., Xiao, L., Sun, Y., Zhang, J., Ma, T., & He, L. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.05.014 · Cited by 800 · see citing works
MOTS: Multi-Object Tracking and Segmentation
Voigtlaender, P., Krause, M., Osep, A., Luiten, J., Sekar, B. B. G., Geiger, A., & Leibe, B. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00813 · Cited by 658 · see citing works
DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation
Wang, G., Zuluaga, M. A., Li, W., Pratt, R., Patel, P. A., Aertsen, M., Doel, T., David, A. L., Deprest, J., Ourselin, S., et al. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2840695 · Cited by 502 · see citing works
Blazingly Fast Video Object Segmentation with Pixel-Wise Metric Learning
Chen, Y., Pont-Tuset, J., Montes, A., & Gool, L. V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00130 · Cited by 273 · see citing works
3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation
Gong, S., Zhong, Y., Ma, W., Li, J., Wang, Z., Zhang, J., Heng, P.-A., & Dou, Q. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103324 · Cited by 156 · see citing works
Luo, X., Wang, G., Song, T., Zhang, J., Aertsen, M., Deprest, J., Ourselin, S., Vercauteren, T., & Zhang, S. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102102 · Cited by 126 · see citing works
Correction-aware interactive 3D tumor segmentation with sparse and revisable prompts
Li, H., & Li, H. (2026) · The Visual Computer · doi:10.1007/s00371-026-04560-5
“Deep interactive object selection introduced click-based distance maps as network inputs [30], while later meth-ods improved iterative refinement by feeding the previous mask back to the network and by designing interaction-aware training protocols [26].”
- Authors
- Xu, N., Price, B., Cohen, S., Yang, J., & Huang, T.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Adobe Systems (United States) (US)
- Venue
- 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2016-06
- Pages
- 373-381
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2300469113
- Cited by
- 460
- References
- 29
- Citations by year
- 2026: 23 · 2025: 56 · 2024: 78 · 2023: 61 · 2022: 51 · 2021: 57 · 2020: 51 · 2019: 36 · 2018: 29 · 2017: 16 · 2016: 2
- FWCI
- 9.60
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- click-based segmentation, objectness, fully convolutional network, unseen object generalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
