CoChat

Long-term recurrent convolutional networks for visual recognition and description

Donahue, Jeff, Hendricks, Lisa Anne, Guadarrama, Sergio, Rohrbach, Marcus, Venugopalan, Subhashini, Darrell, Trevor, Saenko, Kate · 2015 · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.

Abstract

Models based on deep convolutional networks have dominated recent image interpretation tasks; we investigate whether models which are also recurrent, or “temporally deep”, are effective for tasks involving sequences, visual and otherwise. We develop a novel recurrent convolutional architecture suitable for large-scale visual learning which is end-to-end trainable, and demonstrate the value of these models on benchmark video recognition tasks, image description and retrieval problems, and video narration challenges. In contrast to current models which assume a fixed spatio-temporal receptive field or simple temporal averaging for sequential processing, recurrent convolutional models are “doubly deep” in that they can be compositional in spatial and temporal “layers”. Such models may have advantages when target concepts are complex and/or training data are limited. Learning long-term dependencies is possible when nonlinearities are incorporated into the network state updates. Long-term RNN models are appealing in that they directly can map variable-length inputs (e.g., video frames) to variable length outputs (e.g., natural language text) and can model complex temporal dynamics; yet they can be optimized with backpropagation. Our recurrent long-term models are directly connected to modern visual convnet models and can be jointly trained to simultaneously learn temporal dynamics and convolutional perceptual representations. Our results show such models have distinct advantages o…

Overview

Integrity signals

Registry

Crossref · 10.1109/cvpr.2015.7298878

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Research works printed but not linked: 3

  1. simonyan, 0, Very deep convolutional networks for large-scale image recognition, ICLR 2015
  2. kiros, 2014, Multimodal neurallanguage models, ICML
  3. barbu, 2012, Video in sentences out, UAI

Cited by

See all 5,228

Metadata

Authors
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K.
Affiliations
  • University of California, Berkeley (US)
  • International Computer Science Institute (US)
  • The University of Texas at Austin (US)
  • University of Massachusetts Lowell (US)
Venue
2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2015-06
Pages
2625-2634
ISSN
1063-6919 (ISSN-L)
OpenAlex
W1947481528
Cited by
5,228
References
49
Citations by year
2026: 149 · 2025: 246 · 2024: 274 · 2023: 377 · 2022: 412 · 2021: 607 · 2020: 633 · 2019: 758 · 2018: 749 · 2017: 607 · 2016: 305 · 2015: 99
FWCI
238.45
Type
conference paper
Language
en
Open access
closed
Locations
IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
Funders
  • National Science Foundation
  • Nvidia
  • Defense Advanced Research Projects Agency
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
Keywords
recurrent convolutional networks, video recognition, image description generation, image retrieval, video narration, long-range dependencies, temporal dynamics, end-to-end learning, spatiotemporal receptive fields, visual representation learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.