Cited by 236Advanced Neural Network ApplicationsMultimodal Machine Learning ApplicationsVisual Attention and Saliency Detection

Iterative Visual Reasoning Beyond Convolutions

Open access
Chen, Xinlei, Li, Li-Jia, Fei-Fei, Li, Gupta, Abhinav·2018·2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Abstract

We present a novel framework for iterative visual reasoning. Our framework goes beyond current recognition systems that lack the capability to reason beyond stack of convolutions. The framework consists of two core modules: a local module that uses spatial memory [4] to store previous beliefs with parallel updates; and a global graph-reasoning module. Our graph module has three components: a) a knowledge graph where we represent classes as nodes and build edges to encode different types of semantic relationships between them; b) a region graph of the current image where regions in the image are nodes and spatial relationships between these regions are edges; c) an assignment graph that assigns regions to classes. Both the local module and the global module roll-out iteratively and cross-feed predictions to each other to refine estimates. The final predictions are made by combining the best of both modules with an attention mechanism. We show strong performance over plain ConvNets, e.g. achieving an 8.4% absolute improvement on ADE [55] measured by per-class average precision. Analysis also shows that the framework is resilient to missing regions for reasoning.

Integrity signals

Registry

Crossref · 10.1109/cvpr.2018.00756

Cites

56 of 56 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Chen, X., Li, L.-J., Fei-Fei, L., & Gupta, A.
Affiliations
  • Carnegie Mellon University (US)
  • Google (United States) (US)
Venue
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Publisher
IEEE
Published
2018-06
Pages
7239-7248
ISSN
1063-6919 (ISSN-L)
OpenAlex
W2964094751
Cited by
236
References
56
Citations by year
2026: 4 · 2025: 7 · 2024: 15 · 2023: 30 · 2022: 31 · 2021: 48 · 2020: 55 · 2019: 38 · 2018: 8
FWCI
11.23
Type
conference paper
Language
en
Open access
Locations
  • IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Multidisciplinary University Research Initiative
  • Office of Naval Research (N00014-16-1-2007)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
Keywords
spatial memory, graph neural networks, knowledge graphs, region graph, attention mechanism

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review