The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
Segmentation-Aware Convolutional Networks Using Local Attention Masks
Open accessAbstract
We introduce an approach to integrate segmentation information within a convolutional neural network (CNN). This counter-acts the tendency of CNNs to smooth information across regions and increases their spatial precision. To obtain segmentation information, we set up a CNN to provide an embedding space where region co-membership can be estimated based on Euclidean distance. We use these embeddings to compute a local attention mask relative to every neuron position. We incorporate such masks in CNNs and replace the convolution operation with a “segmentation-aware” variant that allows a neuron to selectively attend to inputs coming from its own region. We call the resulting network a segmentation-aware CNN because it adapts its filters at each image point according to local segmentation cues, while at the same time remaining fully-convolutional. We demonstrate the merit of our method on two widely different dense prediction tasks, that involve classification (semantic segmentation) and regression (optical flow). Our results show that in semantic segmentation we can replace DenseCRF inference with a cascade of segmentation-aware filters, and in optical flow we obtain clearly sharper responses than the ones obtained with comparable networks that do not use segmentation. In both cases segmentation-aware convolution yields systematic improvements over strong baselines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“DeepLabV2 is a fullyconvolutional version of a 101-layer residual network (ResNet) [24], modified with atrous spatial pyramid pooling and multi-scale input processing.”
BackgroundVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 1×
“This progress has been built on the remarkable success of CNNs in image classification tasks [29, 50] – indeed, most dense prediction models are based closely on architectures that were successful in object recognition.”
BackgroundImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“This progress has been built on the remarkable success of CNNs in image classification tasks [29, 50] – indeed, most dense prediction models are based closely on architectures that were successful in object recognition.”
BackgroundMicrosoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Fully convolutional networks for semantic segmentationPreprint
Long, J., Shelhamer, E., & Darrell, T. (2014) · Computer Vision and Pattern Recognition · doi:10.48550/arxiv.1605.06211 · Cited by 43,601 · cited 3×
“For example, one effective approach is to add trainable upsampling stages to the network, via “deconvolution” layers [39, 34].”
BackgroundMethodImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 2×
“Both networks are initial- ized with weights learned on ImageNet [46], then trained on the Microsoft COCO training and validation sets [33], and finally fine-tuned on the PASCAL images [16, 21].”
MethodFully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392 · cited 5×
“Experiments are carried out with two different baseline networks, “DeepLab” [9] and “DeepLabV2” [10].”
MethodInfluentialSelf-citationChen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392 · cited 5×
“Experiments are carried out with two different baseline networks, “DeepLab” [9] and “DeepLabV2” [10].”
MethodInfluentialSelf-citationNormalized cuts and image segmentation
Shi, J., & Malik, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.868688 · Cited by 15,935
Flagged first, then by citations
Research works printed but not linked: 1
- everingham, 2012, The pascal visual object classes (voc) challenge 2012 (voc 2012) results
Cited by
Sorted by citations
Items 101–136 of 136
Sharma, V., Choubisa, M., & Shekhawat, S. S. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3356216 · Cited by 4
Bio-Inspired Adaptive Neurons for Dynamic Weighting in Artificial Neural NetworksPreprint
Islam, A., Bouzerdoum, A., & Belhouari, S. B. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5111275 · Cited by 4
Smart manufacturing control system based on deep reinforcement learning
Nikolaev, E., Zakharova, N. I., & Zakharov, V. V. (2021) · IOP Conference Series Materials Science and Engineering · doi:10.1088/1757-899x/1155/1/012061 · Cited by 3
“Deep neural networks have shown high efficiency in the tasks of classification [11-13], object detection [14-16] and segmentation [17-19].”
Kim, S.-H., Nam, H., & Park, Y. (2023) · IEEE Access · doi:10.1109/access.2023.3286034 · Cited by 4
“with minimal increase in computation [56], [57], [58], [59].”
BackgroundFacial Image Completion Using Bi-Directional Pixel LSTM
Yu, X., He, J., & Zhang, Z. (2020) · IEEE Access · doi:10.1109/access.2020.2975827 · Cited by 4
“A multi-scale neural patch synthesis approach was developed in [18].”
MethodRegion Hiding for Image Inpainting via Single-Image Training of U-Net
Martin-King, C., & Allali, M. (2019) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci49370.2019.00117 · Cited by 3
“The authors of [3] refer to the combination of segmentation-aware convolution [20] and automatic mask updating as the partial convolutional layer.”
BackgroundRashmi, R., Prasad, K., & Udupa, C. B. K. (2021) · Communications in computer and information science · doi:10.1007/978-981-16-1086-8_13 · Cited by 2
Learning Depth from Single Image Using Depth-Aware Convolution and Stereo Knowledge
Wu, Z., Wu, X., Zhang, X.-P. S., Wang, S., & Ju, L. (2021) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme51207.2021.9428202 · Cited by 2
“Inspired by the work of [11] which proposes a segmentation-aware CNN by adapting its filters at each pixel based on segmentation cues, we design a novel depth-aware convolution operation for single-view depth estimation based on depth cues.”
MethodLearning Task-Specific Generalized Convolutions in the Permutohedral Lattice
Wannenwetsch, A. S., Kiefel, M., Gehler, P., & Roth, S. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33676-9_24 · Cited by 4
Efficient Instance Segmentation Network
Huang, C., Wu, W., & Lei, Z. (2020) · International Conference on Artificial Intelligence and Information Systems (ICAIIS) · doi:10.1109/icaiis49377.2020.9194856 · Cited by 1
“These kinds of methods first produce pixel-wise classification results, then differentiate different instances[25][26][27][28][29][30].”
MethodOptimal Transport with Arbitrary Prior for Dynamic Resolution Network
Zhang, Z., Li, S., Zhang, C., Ma, L., Tan, X., & Xie, Y. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02483-7 · Cited by 1
Shanmugasundaram, S., & Palaniappan, N. (2023) · International Journal of Image and Graphics · doi:10.1142/s0219467824500542 · Cited by 2
CSDANet: a new lightweight fault diagnosis framework towards heavy noise and small samples
Xu, Z., Jia, Z., Wei, Y., Zhang, S., Jin, Z., & Dong, W. (2024) · Cluster Computing · doi:10.1007/s10586-024-04451-1 · Cited by 1
Park, J., Koh, J. W., Kim, J., Yoon, T., & Jeong, J. (2025) · Results in Engineering · doi:10.1016/j.rineng.2025.107413 · Cited by 1
Dense Attention Convolutional Network for Image Classification
Zhang, H., Qin, K., Zhang, Y., Li, Z., & Xu, K. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1651/1/012184 · Cited by 1
“Visual attention modules have been extensively investigated in the past few years[14,16,17].”
BackgroundMethodBackground Object Removal and Image Inpainting to Fill Irregular Holes
Malhotra, A. J., Chopra, A., Dahiya, R., Yadav, P., & Singhal, A. (2021) · Lecture notes in electrical engineering · doi:10.1007/978-981-16-4149-7_25 · Cited by 1
Near-infrared fusion for deep lightness enhancement
Wang, L., Wang, T., Yang, D., Fang, X., & Wan, S. (2022) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-022-01716-2 · Cited by 1
Scene Text Detection Using Pyramid-Based Text Proposal Network and Transformation Component Network
Praneel, A. S. V., & Rao, D. T. S. (2023) · Indian Journal of Computer Science and Engineering · doi:10.21817/indjcse/2023/v14i1/231401007 · Cited by 1
Pixel Adaptive Filtering UnitsPreprint
Kokkinos, F., Marras, I., Maggioni, M., Slabaugh, G., & Zafeiriou, S. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.10581 · Cited by 3
“With the advent of deep learning, many of these spatially-varying techniques were adopted in a differentiable manner and used as layers in existing neural networks [8, 4, 13, 37].”
Unpaired Image Translation via Adaptive Convolution-based NormalizationPreprint
Cho, W., Kim, K., Kim, E., Kim, H. J., & Choo, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.13271 · Cited by 1
“Recent studies proposed to apply adaptive convolution to a variety of tasks such as semantic segmentation [21, 22] and motion prediction [23].”
Deep auxiliary learning for visual localization using colorization taskPreprint
Tian, M., Nie, Q., Shen, H., & Xia, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.00222 · Cited by 1
“Attention mechanism is widely applied in CNN-based algorithms due to general performance improvements in variety of tasks, from object detection [30], [31], semantic segmentation [32], [33] to image captioning [34], [35].”
BackgroundMejri, M., Mejri, A., Mejri, O., & Fekih, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.03287 · Cited by 1
“Finally, other algorithms first perform semantic segmentation followed by boundary detection[3], pixel clustering[4, 5] or learn an embedding to form instance masks[6, 7, 8, 9].”
Martin-King, C. (2019) · doi:10.36837/chapman.000114
“The inspiration for utilizing masked convolutions and renormalization comes from segmentation-aware convolutional networks [46].”
BackgroundImage Repair Methods Based on Deep Residual Networks
Deng, H., Lin, Z., Li, J., Yao, M., Wang, T., & Luo, H. (2021) · Lecture notes in electrical engineering · doi:10.1007/978-981-16-6554-7_18
Background-Click Supervision for Temporal Action LocalizationPreprint
Yang, L., Han, J., Zhao, T., Lin, T., Zhang, D., & Chen, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.12449
“The proposed affinity module is related to embedding learning [49], [50], [51].”
BackgroundDeep Neural Network Approach Based Segmentation, Detection and Classification of Brain Tumor
Zaman, A., Yu, L., Din, N. U., Ullah, K., & Hayat, Q. (2022) · Journal of Engineering Research and Reports · doi:10.9734/jerr/2022/v22i917563
“Convolution Neural Networks (CNN) approachbased segmentation [12] was applied to natural montage labeling in computer vision.”
MethodSwitchable Representation Learning Framework with Self-Compatibility
Wu, S., Bai, Y., Lou, Y., Linghu, X., He, J., & Duan, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01530
“In recent years, research in this field has mainly focused on dynamic depth [5, 18, 26, 54, 55, 58], dynamic width [3, 4, 11,27,29,36], dynamic routing [22,25,28,64], dynamic parameters [12,15,21,51,69], etc.”
BackgroundThe Dynamic Paradox: How Layer-skipping DNNs Amplify Cache Side-Channel Leakages
She, J., Wang, W., & Wang, Z. (2023) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom60117.2023.00029
“These mechanisms include early-exit [13], [14], soft attention [15], [16], tree structures [17], [18], and layer-skipping [19], [20] etc.”
BackgroundIdentification of Microbial Species Using an Instance Segmentation Model
Davis, C., Lange, A., Blaszczyk, S., Heldt, H., Schaaf, N. A. V., & Manjarrés, J. (2024) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm62325.2024.10822108
ESCAN: Efficient GPU sharing for cascade neural network inference
Wang, J., Shi, Y., Chen, Z., & Wen, M. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107703
Hierarchical Adaptive Attention and Multi-Scale Transformer for Ghost- Free HDR ImagingPreprint
WEN, Z., Gong, S., Qiu, C., & Gao, W. (2025) · Research Square · doi:10.21203/rs.3.rs-6962959/v1
Parameters-adaptive mechanism based on degradation degree for nonhomogeneous image dehazing
Zhang, Z., Zhang, B., Cheng, L., Xu, X., & Ren, M. (2025) · Optics and Lasers in Engineering · doi:10.1016/j.optlaseng.2025.109454
Deep Learning Driven Physiological Signal Generation: An Improved DDIM-Based ECG Synthesis Framework
Wang, Y., Li, Y. F., Li, Y., Sun, Y., Chen, M., & Wei, S. (2025) · International Congress on Image and Signal Processing (CISP) · doi:10.1109/cisp-bmei68103.2025.11259385
“In medical image and signal generation, various strategies such as region supervision, segmentation-guided generation [14], and feature-constrained objectives have been employed to enhance the fidelity of localized structures.”
Spatial techniques for video understanding
Sydorov, V. (2021) · theses.fr (ABES) · doi:10.70675/ab9825adz1fb8z4f1cz9217zef67d5530022
Designing Deep-Learning models for surface and atmospheric retrievals from the IASI infrared sounder
Boucher, E. (2024) · theses.fr (ABES) · doi:10.70675/bd973b10z4931z4814za8e6z9104797a90dd
Introducing a New Partial Convolutional Neural Network for IASI Cloud Classification
Boucher, E., Aires, F., & Doutriaux, M. (2026) · Journal of Geophysical Research Atmospheres · doi:10.1029/2025jd045546
“Such layers were first introduced in (Harley et al., 2017; Liu et al., 2018), but they are exploited here for the first time as a tool to deal with missing data present in the input images.”
Metadata
- Authors
- Harley, A. W., Derpanis, K. G., & Kokkinos, I.
- Affiliations
- Carnegie Mellon University (US)
- Toronto Metropolitan University (CA)
- Meta (United States) (US)
- Venue
- 2017 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 5048-5057
- OpenAlex
- W2747768235
- Cited by
- 142
- References
- 67
- Citations by year
- 2026: 1 · 2025: 7 · 2024: 8 · 2023: 22 · 2022: 14 · 2021: 26 · 2020: 20 · 2019: 28 · 2018: 16
- FWCI
- 5.04
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- UCL Discovery (University College London) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- adaptive filtering, semantic segmentation, optical flow, dense prediction, spatial precision, fully convolutional network
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
