Cited by 142Advanced Neural Network ApplicationsVisual Attention and Saliency DetectionDomain Adaptation and Few-Shot Learning

Segmentation-Aware Convolutional Networks Using Local Attention Masks

Open access
Harley, Adam W., Derpanis, Konstantinos G., Kokkinos, Iasonas·2017·2017 IEEE International Conference on Computer Vision (ICCV)

Abstract

We introduce an approach to integrate segmentation information within a convolutional neural network (CNN). This counter-acts the tendency of CNNs to smooth information across regions and increases their spatial precision. To obtain segmentation information, we set up a CNN to provide an embedding space where region co-membership can be estimated based on Euclidean distance. We use these embeddings to compute a local attention mask relative to every neuron position. We incorporate such masks in CNNs and replace the convolution operation with a “segmentation-aware” variant that allows a neuron to selectively attend to inputs coming from its own region. We call the resulting network a segmentation-aware CNN because it adapts its filters at each image point according to local segmentation cues, while at the same time remaining fully-convolutional. We demonstrate the merit of our method on two widely different dense prediction tasks, that involve classification (semantic segmentation) and regression (optical flow). Our results show that in semantic segmentation we can replace DenseCRF inference with a cascade of segmentation-aware filters, and in optical flow we obtain clearly sharper responses than the ones obtained with comparable networks that do not use segmentation. In both cases segmentation-aware convolution yields systematic improvements over strong baselines.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/iccv.2017.539

Cites

70 of 70 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 4

  1. [16]16. M. Everingham, L. Van-Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. http://www.pascal- network.org/challenges/VOC/voc2012/workshop/index.html, 2012. 6, 7
  2. [22]22. B. Hariharan, P. Arbeláez, R. Girshick, and J. Malik. Hyper- columns for object segmentation and fine-grained localiza- tion. In CVPR, 2015. 3
  3. [32]32. M. Leordeanu, R. Sukthankar, and C. Sminchisescu. Ef- ficient closed-form solution to generalized boundary detec- tion. In ECCV, pages 516-529, 2012. 2
  4. [52]52. S. Sukhbaatar, J. Weston, R. Fergus, et al. End-to-end mem- ory networks. In NIPS, pages 2440-2448, 2015. 2

Cited by

Sorted by citations

Authors
Harley, A. W., Derpanis, K. G., & Kokkinos, I.
Affiliations
  • Carnegie Mellon University (US)
  • Toronto Metropolitan University (CA)
  • Meta (United States) (US)
Venue
2017 IEEE International Conference on Computer Vision (ICCV)
Publisher
IEEE
Published
2017-10
Pages
5048-5057
OpenAlex
W2747768235
Cited by
142
References
67
Citations by year
2026: 1 · 2025: 7 · 2024: 8 · 2023: 22 · 2022: 14 · 2021: 26 · 2020: 20 · 2019: 28 · 2018: 16
FWCI
5.04
Type
conference paper
Language
en
Open access
Locations
  • IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • UCL Discovery (University College London) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
adaptive filtering, semantic segmentation, optical flow, dense prediction, spatial precision, fully convolutional network

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review