The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
Segmentation-Aware Convolutional Networks Using Local Attention Masks
Open accessAbstract
We introduce an approach to integrate segmentation information within a convolutional neural network (CNN). This counter-acts the tendency of CNNs to smooth information across regions and increases their spatial precision. To obtain segmentation information, we set up a CNN to provide an embedding space where region co-membership can be estimated based on Euclidean distance. We use these embeddings to compute a local attention mask relative to every neuron position. We incorporate such masks in CNNs and replace the convolution operation with a “segmentation-aware” variant that allows a neuron to selectively attend to inputs coming from its own region. We call the resulting network a segmentation-aware CNN because it adapts its filters at each image point according to local segmentation cues, while at the same time remaining fully-convolutional. We demonstrate the merit of our method on two widely different dense prediction tasks, that involve classification (semantic segmentation) and regression (optical flow). Our results show that in semantic segmentation we can replace DenseCRF inference with a cascade of segmentation-aware filters, and in optical flow we obtain clearly sharper responses than the ones obtained with comparable networks that do not use segmentation. In both cases segmentation-aware convolution yields systematic improvements over strong baselines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“DeepLabV2 is a fullyconvolutional version of a 101-layer residual network (ResNet) [24], modified with atrous spatial pyramid pooling and multi-scale input processing.”
BackgroundVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“This progress has been built on the remarkable success of CNNs in image classification tasks [29, 50] – indeed, most dense prediction models are based closely on architectures that were successful in object recognition.”
BackgroundImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“This progress has been built on the remarkable success of CNNs in image classification tasks [29, 50] – indeed, most dense prediction models are based closely on architectures that were successful in object recognition.”
BackgroundMicrosoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Fully convolutional networks for semantic segmentationPreprint
Long, J., Shelhamer, E., & Darrell, T. (2014) · Computer Vision and Pattern Recognition · doi:10.48550/arxiv.1605.06211 · Cited by 43,601 · cited 3×
“For example, one effective approach is to add trainable upsampling stages to the network, via “deconvolution” layers [39, 34].”
BackgroundMethodImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 2×
“Both networks are initial- ized with weights learned on ImageNet [46], then trained on the Microsoft COCO training and validation sets [33], and finally fine-tuned on the PASCAL images [16, 21].”
MethodFully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392 · cited 5×
“Experiments are carried out with two different baseline networks, “DeepLab” [9] and “DeepLabV2” [10].”
MethodInfluentialSelf-citationChen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392 · cited 5×
“Experiments are carried out with two different baseline networks, “DeepLab” [9] and “DeepLabV2” [10].”
MethodInfluentialSelf-citationNormalized cuts and image segmentation
Shi, J., & Malik, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.868688 · Cited by 15,935
Flagged first, then by citations
Research works printed but not linked: 1
- everingham, 2012, The pascal visual object classes (voc) challenge 2012 (voc 2012) results
Cited by
Sorted by citations
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596
Image Inpainting for Irregular Holes Using Partial Convolutions
Liu, G., Reda, F. A., Shih, K. J., Wang, T.-C., Tao, A., & Catanzaro, B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_6 · Cited by 2,419
“The partial convolution at every location, similarly defined in [7], is expressed as:”
BackgroundMethodInfluentialYOLACT: Real-Time Instance Segmentation
Bolya, D., Zhou, C., Xiao, F., & Lee, Y. J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00925 · Cited by 2,536
Huang, Z., Huang, L., Gong, Y., Huang, C., & Wang, X. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00657 · Cited by 1,209
Dynamic Neural Networks: A Survey
Han, Y., Huang, G., Song, S., Yang, L., Wang, H., & Wang, Y. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3117837 · Cited by 848
“For example, segmentation-aware convolutional network [108] applies locally masked convolution to aggregate information with larger weights from”
BackgroundMethodInfluentialVideos as Space-Time Region Graphs
Wang, X., & Gupta, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01228-1_25 · Cited by 793
Exploring Cross-Image Pixel Contrast for Semantic Segmentation
Wang, W., Zhou, T., Yu, F., Dai, J., Konukoglu, E., & Gool, L. V. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00721 · Cited by 604
A survey on instance segmentation: state of the art
Hafiz, A. M., & Bhat, G. M. (2020) · International Journal of Multimedia Information Retrieval · doi:10.1007/s13735-020-00195-x · Cited by 590
YOLACT++ Better Real-Time Instance Segmentation
Bolya, D., Zhou, C., Xiao, F., & Lee, Y. J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3014297 · Cited by 544
“Finally, some methods first perform semantic segmentation followed by boundary detection [19], pixel clustering [20], [21], CRF inference [22], or learn an embedding to form instance masks [23], [24], [25], [26].”
BackgroundMethodNon-Local Recurrent Network for Image RestorationPreprint
Liu, D., Wen, B., Fan, Y., Loy, C. C., & Huang, T. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.02919 · Cited by 557
Rethinking Semantic Segmentation: A Prototype View
Zhou, T., Wang, W., Konukoğlu, E., & Goo, L. V. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00261 · Cited by 426
“With MiT-B0, our model achieves the best performance ( i.e ., 38.5% mIoU) with the smallest number of parameters ( i.e ., 3.7 M).”
InfluentialPixel-Adaptive Convolutional Neural Networks
Su, H., Jampani, V., Sun, D., Gallo, O., Learned-Miller, E., & Kautz, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01142 · Cited by 348
“, motion prediction [47], semantic segmentation [17], and Monte Carlo rendering denoising [4].”
BackgroundIterative Residual Refinement for Joint Optical Flow and Occlusion Estimation
Hur, J., & Roth, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00590 · Cited by 298
“To address this, bilateral filters or local attention maps [21, 24] have been proposed as viable solutions.”
BackgroundA Survey on Deep Learning-Based Change Detection from High-Resolution Remote Sensing Images
Jiang, H., Peng, M., Zhong, Y., Xie, H., Hao, Z., Lin, J.-M., Ma, X., & Hu, X. (2022) · Remote Sensing · doi:10.3390/rs14071552 · Cited by 313
Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning
Mascharka, D., Tran, P., Soklaski, R., & Majumdar, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00519 · Cited by 203
“Several authors [7, 10, 25] have proposed attention mechanisms that a network can use, optionally.”
BackgroundDeep Hierarchical Semantic Segmentation
Li, L., Zhou, T., Wang, W., Li, J., & Yang, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00131 · Cited by 212
“Scholars devised many promising solutions, by enlarging receptive fields [10, 13, 19, 93, 97, 105], building image pyramids [33,46], exploring encoder-decoder architectures [3,13,62], utilizing boundary clues [23,41,99], or incorporating neural attention [25, 32, 35, 40, 42, 73, 84, 106, 108, 113].”
BackgroundContrastive Learning for Label Efficient Semantic Segmentation
Zhao, X., Vemulapalli, R., Mansfield, P. A., Gong, B., Green, B., Shapira, L., & Wu, Y. N. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01045 · Cited by 192
“A few existing works [3, 18, 23, 33] use metric learning based on independent pairwise similarity and dissimilarity losses for the tasks of semantic and instance segmentation.”
MethodRecurrent Pixel Embedding for Instance Grouping
Kong, S., & Fowlkes, C. C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00940 · Cited by 171
Zheng, H., Fu, J., Zha, Z.-J., Luo, J., & Mei, T. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2921876 · Cited by 135
“localization caused by blurred feature maps [22], we design a sub-network named as part rectification network (PRN) to further generate attention rectifications.”
BackgroundMethodAttentional Neural Fields for Crowd Counting
Zhang, A., Yue, L., Shen, J., Zhu, F., Zhen, X., Cao, X., & Shao, L. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00581 · Cited by 132
“, image segmentation [9] and depth estimation [34].”
BackgroundSemantics for Robotic Mapping, Perception and Interaction: A Survey
Garg, S., Sünderhauf, N., Dayoub, F., Morrison, D., Cosgun, A., Carneiro, G., Wu, Q., Chin, T.-J., Reid, I. D., Gould, S. J., et al. (2020) · Foundations and Trends in Robotics · doi:10.1561/2300000059 · Cited by 122
Geometry-Aware Distillation for Indoor Semantic Segmentation
Jiao, J., Wei, Y., Jie, Z., Shi, H., Lau, R. W. H., & Huang, T. S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00298 · Cited by 105
Pattern-Structure Diffusion for Multi-Task Learning
Zhou, L., Cui, Z., Xu, C., Zhang, Z., Wang, C., Zhang, T., & Yang, J. W. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00457 · Cited by 97
“Later, many methods [7, 20, 31] are based on FCN.”
MethodKnow What Your Neighbors Do : 3D Semantic Segmentation of Point Clouds
Engelmann, F., Kontogianni, T., Schult, J., & Leibe, B. (2019) · RWTH Publications (RWTH Aachen) · Cited by 96
A Dynamic Multi-Scale Voxel Flow Network for Video Prediction
Hu, X., Huang, Z., Huang, A., Xu, J., & Zhou, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00593 · Cited by 101
Semi-convolutional Operators for Instance Segmentation
Novotný, D., Albanie, S., Larlus, D., & Vedaldi, A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_6 · Cited by 85
Cross-Image Pixel Contrasting for Semantic Segmentation
Zhou, T., & Wang, W. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367952 · Cited by 107
“…to merge multi-layer features [11], [12], [52], applying CRF to recover detailed structures [53], [54], introducing attention mechanisms [55], [56] on the higher layers of FCNs to directly exchange context between paired pixels [13], [14], [57], [58], [59], [60], or designing fully attentive…”
BackgroundMethodAerialFormer: Multi-Resolution Transformer for Aerial Image Segmentation
Hanyu, T., Yamazaki, K., Tran, M., McCann, R., Liao, H., Rainwater, C. E., Adkins, M., Cothren, J. D., & Le, N. (2024) · Remote Sensing · doi:10.3390/rs16162930 · Cited by 104
Prototype-Based Semantic Segmentation
Zhou, T., & Wang, W. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3387116 · Cited by 89
“…[43], [47], [49], [50], [51], [52], [53], [54], [55], [56], [57], [58], [59], [60], [61], [62], [63], [64]; incorporating non-local operations [6], [65], [66], [67], [68], [69], [70], [71], [72], [73], [74], [75]; leveraging boundary information [76], [77], [78], [79], [80], [81], [82]; or…”
GaitGCI: Generative Counterfactual Intervention for Gait Recognition
Dou, H., Zhang, P., Su, W., Yu, Y., Lin, Y., & Li, X. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00540 · Cited by 83
“Besides, weight adjustment could be performed by soft attention over the spatial dimension of the convolutional weights [2, 52, 72].”
Background-Click Supervision for Temporal Action Localization
Yang, L., Han, J., Zhao, T., Lin, T., Zhang, D., & Chen, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3132058 · Cited by 80
“The proposed affinity module is related to embedding learning [49], [50], [51].”
BackgroundRNON: image inpainting via repair network and optimization network
Chen, Y., Xia, R., Zou, K., & Yang, K. (2023) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-023-01811-y · Cited by 75
“[13] and the U-Net structure proposed by Olaf et al .”
Learning to Measure Change: Fully Convolutional Siamese Metric Networks for Scene Change DetectionPreprint
Guo, E., Fu, X., Zhu, J., Deng, M., Liu, Y., Zhu, Q., & Li, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.09111 · Cited by 93
Diverse Semantic Image Synthesis via Probability Distribution Modeling
Tan, Z., Chai, M., Chen, D., Liao, J., Chu, Q., Liu, B., Hua, G., & Yu, N. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00787 · Cited by 70
Ye, Z., Fu, Y., Gan, M., Deng, J., Comber, A., & Wang, K. (2019) · Remote Sensing · doi:10.3390/rs11242970 · Cited by 70
Video Object Segmentation by Learning Location-Sensitive Embeddings
Ci, H., Wang, C., & Wang, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01252-6_31 · Cited by 62
“This has been extensively explored in the area of clustering [28], human pose estimation [23], segmentation [11] and instance segmentation [8, 4].”
BackgroundSegmenting and tracking cell instances with cosine embeddings and recurrent hourglass networks
Payer, C., Štern, D., Feiner, M., Bischof, H., & Urschler, M. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.06.015 · Cited by 64
Partial Convolution for Padding, Inpainting, and Image Synthesis
Liu, G., Dündar, A., Shih, K. J., Wang, T.-C., Reda, F. A., Sapra, K., Yu, Z., Yang, X., Tao, A., & Catanzaro, B. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3209702 · Cited by 66
“Harley et al. [20] use soft attention masks to reweight the convolution results for semantic segmentation.”
BackgroundMetaGait: Learning to Learn an Omni Sample Adaptive Representation for Gait Recognition
Dou, H., Zhang, P., Su, W., Yu, Y., & Li, X. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20065-6_21 · Cited by 57
“Dynamic networks can be mainly divided into dynamic architectures [64,61,49,21,5,39,47] and dynamic parameters [66,12,28,57,14,24,56,15,58].”
BackgroundMedical image synthesis via conditional GANs: Application to segmenting brain tumours
Hamghalam, M., & Simpson, A. L. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.107982 · Cited by 61
RbA: Segmenting Unknown Regions Rejected by All
Nayal, N., Yavuz, M., Henriques, J. F., & Güney, F. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00072 · Cited by 65
“Recent work shifted towards transformer-based architectures [70, 63, 76, 82, 72] and attention mechanisms [29, 43, 81, 30, 44, 37, 22].”
BackgroundKnow What Your Neighbors Do: 3D Semantic Segmentation of Point Clouds
Engelmann, F., Kontogianni, T., Schult, J., & Leibe, B. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11015-4_29 · Cited by 50
“This provides early layers a stronger signal of what should be learned and simplifies gradient propagation as the gradient is passed through fewer layers [1].”
BackgroundDense Convolutional Networks for Semantic Segmentation
Han, C., Duan, Y., Tao, X., & Lü, J. (2019) · IEEE Access · doi:10.1109/access.2019.2908685 · Cited by 51
“Most state-of-the-art methods [9]–[13] consider semantic segmentation as a pixel-wise classification problem and adopt fully convolutional networks (FCNs) [14] to accomplish it.”
LogicSeg: Parsing Visual Semantics with Neural Logic Learning and Reasoning
Li, L., Wang, W., & Yang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00381 · Cited by 49
“Tremendous progress has been achieved by, for example, polishing context cues [36–52], investigating boundary information [53–58], incorporating neural attention [59–71], adopting data structure-aware learning[72–76], and automating network engineering [77–80].”
BackgroundPanoptic-Based Image Synthesis
Dündar, A., Sapra, K., Liu, G., Tao, A., & Catanzaro, B. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00809 · Cited by 46
“We refer to the partial convolution operation using panoptic maps as a Panoptic aware partial convolution layer which shares the fundamentals with other works that use partial convolution for different tasks [7, 15].”
BackgroundMethodDynamic Filtering with Large Sampling Field for ConvNets
Wu, J., Dai, L., Yu, Y., Bajaj, C., & Ji, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01249-6_12 · Cited by 47
“Even though SegAware attention model [10] explicitly takes advantage of boundary information which requires additional training data, our LS-DFN can still slightly outperforms them using FlowNetS as baseline model.”
BackgroundMethodInfluentialSemantic Hierarchy-Aware Segmentation
Li, L., Wang, W., Zhou, T., Quan, R., & Yang, Y. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3332435 · Cited by 49
“…boundary clues [51], [52], [53], [54], considering the co-current relationships between classes [55], [56], [57], incorporating neural attention [58], [59], [60], [61], [62], [63], [64], [65], [66], [67], modeling class distribution [68], [69], [70], and investigating neuro-symbolic computing…”
Stochastic Segmentation with Conditional Categorical Diffusion Models
Zbinden, L., Doorenbos, L., Pissas, T., Huber, A. T., Sznitman, R., & Márquez-Neila, P. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00109 · Cited by 48
“Specifically, the ability to model the spatial distribution of categorical variables is well suited for numerous computer vision tasks, including semantic segmentation [6, 8, 10, 14, 16, 17, 27, 31, 33, 54, 55].”
BackgroundBrain Tumor Synthetic Segmentation in 3D Multimodal MRI Scans
Hamghalam, M., Lei, B., & Wang, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-46640-4_15 · Cited by 42
Dynamic attention network for semantic segmentation
Wu, F., Chen, F., Jing, X., Hu, C., Ge, Q., & Ji, Y. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.12.042 · Cited by 41
ADCNN: Towards learning adaptive dilation for convolutional neural networks
Yao, J., Wang, D., Hu, H., Xing, W., & Wang, L. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108369 · Cited by 40
Hyper-LifelongGAN: Scalable Lifelong Learning for Image Conditioned Generation
Zhai, M., Chen, L., & Mori, G. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00228 · Cited by 40
Does Generative Face Completion Help Face Recognition?
Mathai, J., Masi, I., & AbdAlmageed, W. (2019) · International Conference on Biometrics · doi:10.1109/icb45273.2019.8987388 · Cited by 33
“Though partial convolutions were first introduced in [23], and also used as attention masks in [13], they propose a novel update scheme for the mask that is partially reduced and propagated in subsequent deeper layers.”
BackgroundRethinking Attentive Object Detection via Neural Attention Learning
Ge, C., Song, Y., Ma, C., Qi, Y., & Luo, P. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3251693 · Cited by 33
“…networks have been proven effective in improving the prediction performance in visual perception tasks, including image classification [33], [34], semantic segmentation [29], [35], object tracking [36], [37], [38], [39], [40], person re-identification [41], [42], and action recognition [43], [44].”
BackgroundSSAN: Separable Self-Attention Network for Video Representation Learning
Guo, X., Guo, X., & Lu, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01243 · Cited by 35
“Recently, there have been extensive efforts to study its applications on computer vision tasks such as classification and segmentation [40, 47, 37, 46, 13, 9, 10, 14].”
Rosenfeld, A., Biparva, M., & Tsotsos, J. K. (2018) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00270 · Cited by 29
Deeply Shape-guided Cascade for Instance Segmentation
Ding, H., Qiao, S., Yuille, A., & Shen, W. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00818 · Cited by 29
“…segmentation [1, 43, 44, 21, 14], an increasingly active research topic in recent years, is a combination of the elements from two classical computer vision tasks - object detection [19, 20, 46, 35, 45, 17, 31, 60] and semantic seg-(cid:0) Corresponding Author. mentation [37, 12, 61, 59, 38, 22].”
BackgroundMethodDeep RGB-D Canonical Correlation Analysis For Sparse Depth CompletionPreprint
Zhong, Y., Wu, C.-Y., You, S. O., & Neumann, U. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.08967 · Cited by 28
“SAConv is inspired by local attention mask [34].”
BackgroundMethodMulti-scale YOLACT for instance segmentation
Zeng, J., Ouyang, H., Liu, M., Leng, L., & Fu, X. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2022.09.019 · Cited by 24
DRNet: Double Recalibration Network for Few-Shot Semantic Segmentation
Gao, G., Fang, Z., Han, C., Wei, Y., Liu, C. H., & Yan, S. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3215905 · Cited by 23
Gaur, U., & Manjunath, B. S. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2957937 · Cited by 22
Relevance-Based Compression of Cataract Surgery Videos Using Convolutional Neural Networks
Ghamsarian, N., Amirpour, H., Timmerer, C., Taschwer, M., & Schoeffmann, K. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413658 · Cited by 24
Image Inpainting for Irregular Holes Using Partial ConvolutionsPreprint
Liu, G., Reda, F. A., Shih, K. J., Wang, T.-C., Tao, A., & Catanzaro, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.07723 · Cited by 83
“The partial convolution at every location, similarly defined in [7], is expressed as:”
BackgroundMethodInfluentialAdaMT-Net: An Adaptive Weight Learning Based Multi-Task Learning Model For Scene Understanding
Jha, A., Kumar, A., Banerjee, B., & Chaudhuri, S. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00361 · Cited by 20
“In this regard, convolutional block attention module (CBAM) [22], local attention masks [8], attention U-Net [18], multi-task attention network (MTAN) [16] are some of the popular variants.”
BackgroundGOSS: towards generalized open-set semantic segmentation
Hong, J., Li, W., Han, J., Zheng, J., Fang, P., Harandi, M. T., & Petersson, L. (2023) · The Visual Computer · doi:10.1007/s00371-023-02925-8 · Cited by 23
“Many existing works have utilized similar metric learning techniques for segmentation tasks and gain better empirical results [26, 16, 32, 65].”
BackgroundEfficient Visual Recognition: A Survey on Recent Advances and Brain-inspired Methodologies
Wu, Y., Wang, D., Lu, X., Yang, F., Yao, M., Dong, W., Shi, J., & Li, G. (2022) · Machine Intelligence Research · doi:10.1007/s11633-022-1340-5 · Cited by 22
Image Inpainting for Digital Dunhuang Murals Using Partial Convolutions and Sliding Window Method
Chen, M., Zhao, X., & Xu, D. (2019) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1302/3/032040 · Cited by 18
“Therefore, similar to [15], the partial convolution can be defined as:”
BackgroundPerson Retrieval in Video Surveillance Using Deep Learning–Based Instance Segmentation
Tseng, C.-H., Hsieh, C.-C., Jwo, D., Wu, J.-H., Sheu, R., & Chen, L.-C. (2021) · Journal of Sensors · doi:10.1155/2021/9566628 · Cited by 18
High-Level Task-Driven Single Image Deraining: Segmentation in Rainy Days
Guo, M., Chen, M., Ma, C., Li, Y., Li, X., & Xie, X. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-63830-6_30 · Cited by 18
Segmentation-Aware Image Denoising without Knowing True SegmentationPreprint
Wang, S., Wen, B., Wu, J., Tao, D., & Wang, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.08965 · Cited by 16
“As pointed out by [15], the availability of segmentation information can compromise the over-smoothening effects of CNNs across regions and increases their spatial precision.”
BackgroundStructure-Informed Shadow Removal Networks
Liu, Y., Guo, Q., Fu, L., Ke, Z., Xu, K., Feng, W., Tsang, I. W.-H., & Lau, R. W. H. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3323814 · Cited by 15
“Inspired by the segmentation-aware convolution [46], we”
Can Attention Masks Improve Adversarial Robustness?
Vaishnavi, P., Cong, T., Eykholt, K., Prakash, A., & Rahmati, A. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-62144-5_2 · Cited by 13
Fine-grained Visible Watermark Removal
Niu, L., Zhao, X., Zhang, B., & Zhang, L. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01173 · Cited by 14
“According to [21], dynamic parameters can be classified into parameter adjustment [22, 15], weight prediction [43, 25, 39, 19], and dynamic features [30, 5].”
BackgroundSensIR: Towards privacy-sensitive image retrieval in the cloud
Hu, L., Xiang, T., & Guo, S. (2020) · Signal Processing Image Communication · doi:10.1016/j.image.2020.115837 · Cited by 12
Semantic Probability Distribution Modeling for Diverse Semantic Image Synthesis
Tan, Z., Chu, Q., Chai, M., Chen, D., Liao, J., Liu, Q., Liu, B., Hua, G., & Yu, N. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3210085 · Cited by 12
“An instance aware partial convolution [53], [54] is used to avoid information contamination between different instances.”
MethodVisual Micro-Pattern Propagation
Cui, Z., Zhou, L., Wang, C., Xu, C., & Yang, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3147974 · Cited by 11
“Afterwards, many variants [1], [19], [20] sprung up based on the FCN framework.”
Learning pixel-adaptive weights for portrait photo retouching
Wang, B., Lu, C., Yan, D., Zhao, Y., Li, N., & Li, X. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109775 · Cited by 11
A Dynamic Part-Attention Model for Person Re-Identification
Yao, Z., Wu, X., Xiong, Z., & Ma, Y. (2019) · Sensors · doi:10.3390/s19092080 · Cited by 10
“Some work [29–31] use metric learning to ensure that pixels from the same instance have similar embedding for better segmentation.”
BackgroundPedestrian instance segmentation with prior structure of semantic parts
Chu, H., Ma, H., & Li, X. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.05.012 · Cited by 9
Liu, Z., Gu, D., Zhang, Y., Cao, X., & Xue, Z. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-72084-1_4 · Cited by 9
Multi-Scale Aggregation Network for Direct Face Alignment
Li, P., Zhang, A., Yue, L., Zhen, X., & Cao, X. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00233 · Cited by 8
“Attention mechanism has been introduced into deep learning for several years and achieved great success in many tasks such as person re-identification [39], face hallucination [3], human action recognition [15], segmentation [7] and visual question answering [41] etc.”
BackgroundError Correction for Dense Semantic Image Labeling
Huang, Y.-H., Xu, J., Georgoulis, S., Tuytelaars, T., & Gool, L. V. (2018) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00146 · Cited by 7
Occluded target detection of streak tube imaging lidar using image inpainting
Li, W., Guo, S., Zhai, Y. J., Han, S., Liu, F., & Lai, Z. (2020) · Measurement Science and Technology · doi:10.1088/1361-6501/abd1b6 · Cited by 8
“For the current convolutional window, X is the eigenvalue (pixel value) and M is the corresponding binary mask [17].”
Image inpainting with contextual attention and partial convolution
Mohite, T. A., & Phadke, G. (2020) · International Conference on Artificial Intelligence and Signal Processing (AISP) · doi:10.1109/aisp48273.2020.9073008 · Cited by 8
“Proposed partial convolution for every part, appropriately defined in [8], and equation in terms of expressed as: Here, (cid:12) referred to product considering element to element.”
BackgroundSemi-convolutional Operators for Instance SegmentationPreprint
Novotný, D., Albanie, S., Larlus, D., & Vedaldi, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.10712 · Cited by 7
“It has been used for body joints [41], semantic segmentation [1,21,6] and optical flow [1], and, more relevant to our work, to instance segmentation [15,12,6,28].”
MethodImproving Multiperson Pose Estimation by Mask-aware Deep Reinforcement Learning
Wang, X., Tian, Y. T., Zhao, X., Yang, T., Gelernter, J., Wang, J., Cheng, G., & Hu, W. (2020) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3397340 · Cited by 6
“Most of the approaches multiply the foreground mask by the image [Harley et al. 2017] or the feature maps [Dai et al. 2015].”
MethodRelationRS: Relationship Representation Network for Object Detection in Aerial Images
Liu, Z., Zhang, X., Liu, C., Wang, H., Sun, C., Li, B., Huang, P., Li, Q., Liu, Y., Kuang, H., et al. (2022) · Remote Sensing · doi:10.3390/rs14081862 · Cited by 6
Learning Lightweight Dynamic Kernels With Attention Inside via Local–Global Context Fusion
Tian, Y., Shen, Y., Wang, X., Wang, J., Wang, K., Ding, W., Wang, Z., & Wang, F. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3217301 · Cited by 6
“Toward image generation and segmentation tasks, gated pixel CNN [47] and SegAware [48] propose to learn masks on kernels for local filtering.”
MethodDynamic-HDC: A Two-Stage Dynamic Inference Framework for Brain-Inspired Hyperdimensional Computing
Chuang, Y.-C., Chang, C.-Y., & Wu, A.-Y. A. (2023) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2023.3328857 · Cited by 6
“Note that although several papers have investigated early exit or dynamically adjusting model parameters in DNNs [12], [13], [14], [15], such advantages have been relatively unexplored in the context of HDC.”
BackgroundTowards better semantic consistency of 2D medical image segmentation
Wen, Y., Chen, L., Deng, Y., Ning, J., & Zhou, C. (2021) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2021.103311 · Cited by 6
CBFNet: Constraint balance factor for semantic segmentation
Xu, H., Gao, Y., Li, J., & Gao, X. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.02.039 · Cited by 6
Video Object Segmentation-aware Video Frame Interpolation
Yoo, J., Hong-Jae, L., & Jung, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01132 · Cited by 5
“Harley et al. [22] extracted foreground-focused local features using segmentation-aware CNNs, resulting in significant performance improvements in optical flow estimation and semantic segmentation tasks.”
MethodRealtime single-stage instance segmentation network based on anchors
Cai, J., & Li, Y. (2021) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2021.107464 · Cited by 5
Consensus Feature Network for Scene Parsing
Wu, T., Tang, S., Zhang, R., & Guo, G. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3094333 · Cited by 5
Sharma, H., Pradhan, P., & Balamuralidhar, P. (2021) · Indian Conference on Computer Vision, Graphics and Image Processing · doi:10.1145/3490035.3490281 · Cited by 5
“Recently, attention mechanism was also adopted in various semantic segmentation approaches [46], [20], [26], [53].”
ACMatch: Improving context capture for two-view correspondence learning via adaptive convolution
Fang, X., Lu, Y., Zhang, S., Xie, Y., & Ma, J. (2024) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2024.11.004 · Cited by 4
Asymmetric cost aggregation network for efficient stereo matching
Wu, Z., Zhu, H., He, L., Wang, D., Shi, J., & Wu, W. (2023) · IET Image Processing · doi:10.1049/ipr2.12807 · Cited by 4
Intra-inter Modal Attention Blocks for RGB-D Semantic Segmentation
Choi, S., Zhang, Y., & Hong, S. (2023) · International Conference on Multimedia Retrieval · doi:10.1145/3591106.3592235 · Cited by 4
“The fully convolutional network (FCN) [34] was the first deep learningbased end-to-end network, and it has become the basic structure for many subsequent CNN-based approaches such as [7, 19, 33].”
BackgroundDo End-to-end Stereo Algorithms Under-utilize Information?
Cai, C., & Mordohai, P. (2020) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv50981.2020.00047 · Cited by 4
Fang, Z., Gao, G., Zhang, Z., & Zhang, A. (2023) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2023.103754 · Cited by 4
Metadata
- Authors
- Harley, A. W., Derpanis, K. G., & Kokkinos, I.
- Affiliations
- Carnegie Mellon University (US)
- Toronto Metropolitan University (CA)
- Meta (United States) (US)
- Venue
- 2017 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 5048-5057
- OpenAlex
- W2747768235
- Cited by
- 142
- References
- 67
- Citations by year
- 2026: 1 · 2025: 7 · 2024: 8 · 2023: 22 · 2022: 14 · 2021: 26 · 2020: 20 · 2019: 28 · 2018: 16
- FWCI
- 5.04
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- adaptive filtering, semantic segmentation, optical flow, dense prediction, spatial precision, fully convolutional network
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
