The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
Segmentation-Aware Convolutional Networks Using Local Attention Masks
Open accessAbstract
We introduce an approach to integrate segmentation information within a convolutional neural network (CNN). This counter-acts the tendency of CNNs to smooth information across regions and increases their spatial precision. To obtain segmentation information, we set up a CNN to provide an embedding space where region co-membership can be estimated based on Euclidean distance. We use these embeddings to compute a local attention mask relative to every neuron position. We incorporate such masks in CNNs and replace the convolution operation with a “segmentation-aware” variant that allows a neuron to selectively attend to inputs coming from its own region. We call the resulting network a segmentation-aware CNN because it adapts its filters at each image point according to local segmentation cues, while at the same time remaining fully-convolutional. We demonstrate the merit of our method on two widely different dense prediction tasks, that involve classification (semantic segmentation) and regression (optical flow). Our results show that in semantic segmentation we can replace DenseCRF inference with a cascade of segmentation-aware filters, and in optical flow we obtain clearly sharper responses than the ones obtained with comparable networks that do not use segmentation. In both cases segmentation-aware convolution yields systematic improvements over strong baselines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“DeepLabV2 is a fullyconvolutional version of a 101-layer residual network (ResNet) [24], modified with atrous spatial pyramid pooling and multi-scale input processing.”
BackgroundVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“This progress has been built on the remarkable success of CNNs in image classification tasks [29, 50] – indeed, most dense prediction models are based closely on architectures that were successful in object recognition.”
BackgroundImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“This progress has been built on the remarkable success of CNNs in image classification tasks [29, 50] – indeed, most dense prediction models are based closely on architectures that were successful in object recognition.”
BackgroundMicrosoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Fully convolutional networks for semantic segmentationPreprint
Long, J., Shelhamer, E., & Darrell, T. (2014) · Computer Vision and Pattern Recognition · doi:10.48550/arxiv.1605.06211 · Cited by 43,601 · cited 3×
“For example, one effective approach is to add trainable upsampling stages to the network, via “deconvolution” layers [39, 34].”
BackgroundMethodImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 2×
“Both networks are initial- ized with weights learned on ImageNet [46], then trained on the Microsoft COCO training and validation sets [33], and finally fine-tuned on the PASCAL images [16, 21].”
MethodFully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392 · cited 5×
“Experiments are carried out with two different baseline networks, “DeepLab” [9] and “DeepLabV2” [10].”
MethodInfluentialSelf-citationChen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392 · cited 5×
“Experiments are carried out with two different baseline networks, “DeepLab” [9] and “DeepLabV2” [10].”
MethodInfluentialSelf-citationNormalized cuts and image segmentation
Shi, J., & Malik, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.868688 · Cited by 15,935
Scale-space and edge detection using anisotropic diffusion
Perona, P., & Malik, J. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.56205 · Cited by 12,143
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · cited 2×
“We achieve this by casting the mask creation (i.e., pairwise embedding distance computation) as a convolution-like operation, and implementing it in exactly the way Caffe [26] realizes convolution: via an image-to-column transformation, followed by matrix multiplication.”
MethodBilateral Filtering for Gray and Color Images
Tomasi, C., & Manduchi, R. (2002) · Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271) · doi:10.1109/iccv.1998.710815 · Cited by 8,177 · cited 3×
“The general idea of combining filtering with segmentation to enhance sharpness dates back to nonlinear image processing [42, 53] and segmentation-aware feature extraction [54, 55].”
BackgroundMethodMulti-scale context aggregation by dilated convolutions
Yu, F., & Koltun, V. (2015) · arXiv · doi:10.48550/arxiv.1511.07122 · Cited by 7,738 · cited 16×
“Recently, the CRF approach has been integrated more closely with the CNN, by framing the CRF as a recurrent network, and chaining it to the backpropagation of the underlying CNN [61].”
MethodInfluentialCaffe: Convolutional Architecture for Fast Feature EmbeddingPreprint
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 5,299 · cited 2×
“We achieve this by casting the mask creation (i.e., pairwise embedding distance computation) as a convolution-like operation, and implementing it in exactly the way Caffe [26] realizes convolution: via an image-to-column transformation, followed by matrix multiplication.”
MethodLearning Deconvolution Network for Semantic Segmentation
Noh, H., Hong, S., & Han, B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.178 · Cited by 4,422 · cited 2×
“For example, one effective approach is to add trainable upsampling stages to the network, via “deconvolution” layers [39, 34].”
BackgroundMethodFlownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313 · cited 5×
“Convolutional neural networks (CNNs) have recently made rapid progress in pixel-wise prediction tasks, including depth prediction [15], optical flow estimation [14], and semantic segmentation [47, 9, 34].”
BackgroundMethodInfluentialSemantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213 · cited 20×
“Recently, the CRF approach has been integrated more closely with the CNN, by framing the CRF as a recurrent network, and chaining it to the backpropagation of the underlying CNN [61].”
BackgroundMethodInfluentialSelf-citationDeep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196 · cited 1×
“DeepLabV2 is a fullyconvolutional version of a 101-layer residual network (ResNet) [24], modified with atrous spatial pyramid pooling and multi-scale input processing.”
BackgroundLearning a Similarity Metric Discriminatively, with Application to Face Verification
Chopra, S., Hadsell, R., & LeCun, Y. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.202 · Cited by 4,036 · cited 2×
“Subsequent related work has yielded compelling results for tasks such as wide-baseline stereo correspondence [20, 59, 60], and face verification [11].”
BackgroundDepth Map Prediction from a Single Image using a Multi-Scale Deep Network
Eigen, D., Puhrsch, C., & Fergus, R. (2014) · arXiv · doi:10.48550/arxiv.1406.2283 · Cited by 3,442 · cited 1×
“Convolutional neural networks (CNNs) have recently made rapid progress in pixel-wise prediction tasks, including depth prediction [15], optical flow estimation [14], and semantic segmentation [47, 9, 34].”
BackgroundHolistically-Nested Edge Detection
Xie, S., & Tu, Z. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.164 · Cited by 3,401
Return of the Devil in the Details: Delving Deep into Convolutional Nets
Chatfield, K., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.6 · Cited by 3,335
SUSAN—a new approach to low level image processing
Smith, S. M., & Brady, J. M. (1997) · International Journal of Computer Vision · doi:10.1023/a:1007963824710 · Cited by 3,333 · cited 2×
“• if ej = (i, pi), where pi denotes the color vector at i, the equation yields bilateral filtering [31, 2, 51, 53].”
BackgroundMethodEfficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Krähenbühl, P., & Koltun, V. (2012) · arXiv · doi:10.48550/arxiv.1210.5644 · Cited by 2,908 · cited 14×
“Recently, the CRF approach has been integrated more closely with the CNN, by framing the CRF as a recurrent network, and chaining it to the backpropagation of the underlying CNN [61].”
BackgroundMethodInfluentialOverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680 · cited 1×
“Convolutional neural networks (CNNs) have recently made rapid progress in pixel-wise prediction tasks, including depth prediction [15], optical flow estimation [14], and semantic segmentation [47, 9, 34].”
BackgroundConditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · cited 2×
“Recently, the CRF approach has been integrated more closely with the CNN, by framing the CRF as a recurrent network, and chaining it to the backpropagation of the underlying CNN [61].”
BackgroundMethodSIGNATURE VERIFICATION USING A “SIAMESE” TIME DELAY NEURAL NETWORK
BROMLEY, J., BENTZ, J. W., BOTTOU, L., GUYON, I., LECUN, Y., MOORE, C., SÄCKINGER, E., & SHAH, R. (1994) · Series in Machine Perception and Artificial Intelligence · doi:10.1142/9789812797926_0003 · Cited by 2,350
Conditional Image Generation with PixelCNN DecodersPreprint
van den Oord, A., Kalchbrenner, N., Vinyals, O., Espeholt, L., Graves, A., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.05328 · Cited by 2,032 · cited 1×
“Concurrent work in language modelling [13] and image generation [40] has also emphasized the importance of locally masked (or “gated”) convolutions.”
BackgroundLanguage Modeling with Gated Convolutional NetworksPreprint
Dauphin, Y., Fan, A., Auli, M., & Grangier, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.08083 · Cited by 1,765 · cited 1×
“Concurrent work in language modelling [13] and image generation [40] has also emphasized the importance of locally masked (or “gated”) convolutions.”
BackgroundSemantic contours from inverse detectors
Hariharan, B., Arbeláez, P., Bourdev, L. D., Maji, S., & Malik, J. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126343 · Cited by 1,743 · cited 1×
“ized with weights learned on ImageNet [46], then trained on the Microsoft COCO training and validation sets [33], and finally fine-tuned on the PASCAL images [16, 21].”
MethodDeeply-Supervised NetsPreprint
Lee, C., Xie, S., Gallagher, P., Zhang, Z., & Tu, Z. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.5185 · Cited by 1,693 · cited 1×
“The motivation behind this strategy is to provide early layers a stronger signal of the network’s end goal, reducing the burden on backpropagation to carry the signal through multiple layers [30].”
BackgroundHypercolumns for object segmentation and fine-grained localization
Hariharan, B., Arbelaez, P., Girshick, R., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298642 · Cited by 1,511 · cited 1×
“A complementary approach is to stack features from multiple resolutions near the end of the network, so that the final stages have access to both high-resolution (shallow) features and lowresolution (deep) features [22, 37, 14].”
BackgroundOptical Flow Estimation Using a Spatial Pyramid Network
Ranjan, A., & Black, M. J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.291 · Cited by 1,379 · cited 1×
“SPyNet [44] 2.”
Hierarchical Question-Image Co-Attention for Visual Question AnsweringPreprint
Lu, J., Yang, J., Batra, D., & Parikh, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00061 · Cited by 1,357 · cited 1×
“Similar attention mechanisms are being used in visual [35] and non-visual [52] question answering tasks.”
BackgroundSukhbaatar, S., Szlam, A., Weston, J., & Fergus, R. (2015) · arXiv · doi:10.48550/arxiv.1503.08895 · Cited by 1,345
Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation
Brox, T., & Malik, J. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.143 · Cited by 1,295
Learning to compare image patches via convolutional neural networks
Zagoruyko, S., & Komodakis, N. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299064 · Cited by 1,140 · cited 1×
“Subsequent related work has yielded compelling results for tasks such as wide-baseline stereo correspondence [20, 59, 60], and face verification [11].”
BackgroundDeepFlow: Large Displacement Optical Flow with Deep Matching
Weinzaepfel, P., Revaud, J., Harchaoui, Z., & Schmid, C. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.175 · Cited by 1,057
MatchNet: Unifying feature and metric learning for patch-based matching
Han, X., Leung, T., Jia, Y., Sukthankar, R., & Berg, A. C. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298948 · Cited by 847 · cited 1×
“Subsequent related work has yielded compelling results for tasks such as wide-baseline stereo correspondence [20, 59, 60], and face verification [11].”
BackgroundComputing the stereo matching cost with a convolutional neural network
Žbontar, J., & LeCun, Y. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298767 · Cited by 838 · cited 1×
“Subsequent related work has yielded compelling results for tasks such as wide-baseline stereo correspondence [20, 59, 60], and face verification [11].”
BackgroundDigital image smoothing and the sigma filter
Lee, J.-S. (1983) · Computer Vision Graphics and Image Processing · doi:10.1016/0734-189x(83)90047-6 · Cited by 825
Discriminative Learning of Deep Convolutional Feature Point Descriptors
Simo‐Serra, E., Trulls, E., Ferraz, L., Kokkinos, I., Fua, P., & Moreno-Noguer, F. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.22 · Cited by 818 · cited 1×
“Recently, the topic of metric learning has been studied extensively in conjunction with image descriptors, such as SIFT and SID [54, 49, 3], improving the applicability of those descriptors to patch-matching problems.”
BackgroundSelf-citationAssociative Embedding: End-to-End Learning for Joint Detection and GroupingPreprint
Newell, A., Huang, Z., & Deng, J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05424 · Cited by 808 · cited 1×
“Concurrent work has targeted a similar goal, for body joints [38] and instance segmentation [17].”
BackgroundEpicFlow: Edge-preserving interpolation of correspondences for optical flow
Revaud, J., Weinzaepfel, P., Harchaoui, Z., & Schmid, C. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298720 · Cited by 757 · cited 1×
“63 EpicFlow [45] 2.”
Holistically-nested edge detection
Xie, S., & Tu, Z. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-017-1004-z · Cited by 682 · cited 1×
“The idea of using a loss at intermediate layers is inspired by Xie and Tu [57], who used this strategy to learn boundary cues in a CNN.”
MethodReturn of the Devil in the Details: Delving Deep into Convolutional NetsPreprint
Chatfield, K., Simonyan, K., Vedaldi, A., et al. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.3531 · Cited by 663 · cited 2×
“The first seven layers share the design of the earliest convolution layers in VGG-16 [7], and are initialized with that network’s (object recognition-trained) weights.”
BackgroundMethodLearning Deconvolution Network for Semantic SegmentationPreprint
Noh, H., Hong, S., & Han, B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.04366 · Cited by 613 · cited 2×
“For example, one effective approach is to add trainable upsampling stages to the network, via “deconvolution” layers [39, 34].”
BackgroundMethodFull-Resolution Residual Networks for Semantic Segmentation in Street Scenes
Pohlen, T., Hermans, A., Mathias, M., & Leibe, B. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.353 · Cited by 600 · cited 1×
“The low-resolution issue has received substantial attention: for instance methods have been proposed for replacing the subsampling layers with resolution-preserving alternatives such as atrous convolution [9, 58, 43], or restoring the lost resolution via upsampling stages [39, 34].”
MethodFeedforward semantic segmentation with zoom-out features
Mostajabi, M., Yadollahpour, P., & Shakhnarovich, G. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298959 · Cited by 446 · cited 1×
“A complementary approach is to stack features from multiple resolutions near the end of the network, so that the final stages have access to both high-resolution (shallow) features and lowresolution (deep) features [22, 37, 14].”
BackgroundConvolutional feature masking for joint object and stuff segmentation
Dai, J., He, K., & Sun, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299025 · Cited by 437 · cited 1×
“More in line with the current paper are recent works that incorporates segmentation cues into CNNs, by sharpening or masking intermediate feature maps with the help of superpixels [12, 19].”
BackgroundUsing contours to detect and localize junctions in natural images
Maire, M., Arbeláez, P., Fowlkes, C. C., & Malik, J. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587420 · Cited by 403 · cited 1×
“For instance in [54, 55], soft segmentation masks [41, 32] and boundary cues [36, 48] have been used to develop segmentation-aware variants of handcrafted features, like SIFT and HOG, effectively suppressing contributions from pixels likely to come from the background.”
MethodFast High‐Dimensional Filtering Using the Permutohedral Lattice
Adams, A., Baek, J., & Davis, M. A. (2010) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01645.x · Cited by 394 · cited 1×
“, the size of the image), which necessitates using a data structure called a permutohedral lattice [1].”
MethodChen, L.-C., Barron, J. T., Papandreou, G., Murphy, K. P., & Yuille, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.492 · Cited by 378 · cited 1×
“, using edges found in the image [8, 4], or using a dense conditional random field (CRF) [28, 9, 58].”
MethodFrome, A., Singer, Y., Sha, F., & Malik, J. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4408839 · Cited by 348 · cited 1×
“The goal of metric learning is to produce features from which one can estimate the similarity between pixels or regions in the input [18].”
BackgroundNon-Linear Gaussian Filters Performing Edge Preserving Diffusion
Aurich, V., & Weule, J. (1995) · Informatik aktuell · doi:10.1007/978-3-642-79980-8_63 · Cited by 303 · cited 2×
“• if ej = (i, pi), where pi denotes the color vector at i, the equation yields bilateral filtering [31, 2, 51, 53].”
BackgroundMethodNormalized and differential convolution
Knutsson, H. E., & Westin, C.-F. (2002) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1993.341081 · Cited by 245 · cited 1×
“The mask acts as an applicability function for the filter, which makes segmentation-aware convolution a special case of normalized convolution [27].”
BackgroundSemantic Instance Segmentation via Deep Metric LearningPreprint
Fathi, A., Wojna, Z., Rathod, V., Wang, P., Song, H. O., Guadarrama, S., & Murphy, K. P. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.10277 · Cited by 213 · cited 1×
“Concurrent work has targeted a similar goal, for body joints [38] and instance segmentation [17].”
BackgroundSemantic Segmentation with Boundary Neural Fields
Bertasius, G., Shi, J., & Torresani, L. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.392 · Cited by 209 · cited 1×
“, using edges found in the image [8, 4], or using a dense conditional random field (CRF) [28, 9, 58].”
MethodLearning Sparse High Dimensional Filters: Image Filtering, Dense CRFs and Bilateral Neural Networks
Jampani, V., Kiefel, M., & Gehler, P. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.482 · Cited by 192 · cited 1×
“[25] propose to learn the kernel used in the bilateral filter, but keep the arguments to the similarity measure (i.”
BackgroundPN-Net: Conjoined Triple Deep Network for Learning Local Image DescriptorsPreprint
Balntas, V., Johns, E., Tang, L., & Mikolajczyk, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1601.05030 · Cited by 174
Learning to Compare Image Patches via Convolutional Neural NetworksPreprint
Zagoruyko, S., & Nikos Komodakis, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.03641 · Cited by 161 · cited 1×
“Subsequent related work has yielded compelling results for tasks such as wide-baseline stereo correspondence [20, 59, 60], and face verification [11].”
BackgroundSuperpixel Convolutional Networks Using Bilateral Inceptions
Gadde, R., Jampani, V., Kiefel, M., Kappler, D., & Gehler, P. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46448-0_36 · Cited by 130 · cited 1×
“More in line with the current paper are recent works that incorporates segmentation cues into CNNs, by sharpening or masking intermediate feature maps with the help of superpixels [12, 19].”
BackgroundEfficient Closed-Form Solution to Generalized Boundary Detection
Leordeanu, M., Sukthankar, R., & Sminchisescu, C. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33765-9_37 · Cited by 85 · cited 1×
“For instance in [54, 55], soft segmentation masks [41, 32] and boundary cues [36, 48] have been used to develop segmentation-aware variants of handcrafted features, like SIFT and HOG, effectively suppressing contributions from pixels likely to come from the background.”
MethodImplicit color segmentation features for pedestrian and object detection
Ott, P., & Everingham, M. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459238 · Cited by 75 · cited 1×
“For instance in [54, 55], soft segmentation masks [41, 32] and boundary cues [36, 48] have been used to develop segmentation-aware variants of handcrafted features, like SIFT and HOG, effectively suppressing contributions from pixels likely to come from the background.”
MethodDense Segmentation-Aware Descriptors
Trulls, E., Kokkinos, I., Sanfeliu, A., & Moreno-Noguer, F. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.372 · Cited by 51 · cited 3×
“The general idea of combining filtering with segmentation to enhance sharpness dates back to nonlinear image processing [42, 53] and segmentation-aware feature extraction [54, 55].”
BackgroundMethodSelf-citationFeedforward semantic segmentation with zoom-out featuresPreprint
Mostajabi, M., Yadollahpour, P., & Shakhnarovich, G. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.0774 · Cited by 32 · cited 1×
“A complementary approach is to stack features from multiple resolutions near the end of the network, so that the final stages have access to both high-resolution (shallow) features and lowresolution (deep) features [22, 37, 14].”
BackgroundSegmentation-Aware Deformable Part Models
Trulls, E., Tsogkas, S., Kokkinos, I., Sanfeliu, A., & Moreno-Noguer, F. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.29 · Cited by 29 · cited 2×
“The general idea of combining filtering with segmentation to enhance sharpness dates back to nonlinear image processing [42, 53] and segmentation-aware feature extraction [54, 55].”
BackgroundMethodSelf-citationLearning Dense Convolutional Embeddings for Semantic SegmentationPreprint
Harley, A. W., Derpanis, K. G., & Kokkinos, I. (2015) · HAL · doi:10.48550/arxiv.1511.04377 · Cited by 21 · cited 2×
“In contrast, the focus of our work is as in [23] to bring a given point close to all of the other points that lie in the same object.”
BackgroundMethodSelf-citation
Research works printed but not linked: 1
- everingham, 2012, The pascal visual object classes (voc) challenge 2012 (voc 2012) results
Cited by
Sorted by citations
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596
Image Inpainting for Irregular Holes Using Partial Convolutions
Liu, G., Reda, F. A., Shih, K. J., Wang, T.-C., Tao, A., & Catanzaro, B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_6 · Cited by 2,419
“The partial convolution at every location, similarly defined in [7], is expressed as:”
BackgroundMethodInfluentialYOLACT: Real-Time Instance Segmentation
Bolya, D., Zhou, C., Xiao, F., & Lee, Y. J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00925 · Cited by 2,536
Huang, Z., Huang, L., Gong, Y., Huang, C., & Wang, X. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00657 · Cited by 1,209
Dynamic Neural Networks: A Survey
Han, Y., Huang, G., Song, S., Yang, L., Wang, H., & Wang, Y. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3117837 · Cited by 848
“For example, segmentation-aware convolutional network [108] applies locally masked convolution to aggregate information with larger weights from”
BackgroundMethodInfluential
Metadata
- Authors
- Harley, A. W., Derpanis, K. G., & Kokkinos, I.
- Affiliations
- Carnegie Mellon University (US)
- Toronto Metropolitan University (CA)
- Meta (United States) (US)
- Venue
- 2017 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 5048-5057
- OpenAlex
- W2747768235
- Cited by
- 142
- References
- 67
- Citations by year
- 2026: 1 · 2025: 7 · 2024: 8 · 2023: 22 · 2022: 14 · 2021: 26 · 2020: 20 · 2019: 28 · 2018: 16
- FWCI
- 5.04
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- adaptive filtering, semantic segmentation, optical flow, dense prediction, spatial precision, fully convolutional network
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
