CoChat

Asynchronous Temporal Fields for Action Recognition

Sigurdsson, Gunnar A., Divvala, Santosh, Farhadi, Ali, Gupta, Abhinav · 2017 · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Cited by 144

The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.

Abstract

Actions are more than just movements and trajectories: we cook to eat and we hold a cup to drink from it. A thorough understanding of videos requires going beyond appearance modeling and necessitates reasoning about the sequence of activities, as well as the higher-level constructs such as intentions. But how do we model and reason about these? We propose a fully-connected temporal CRF model for reasoning over various aspects of activities that includes objects, actions, and intentions, where the potentials are predicted by a deep network. End-to-end training of such structured models is a challenging endeavor: For inference and learning we need to construct mini-batches consisting of whole videos, leading to mini-batches with only a few videos. This causes high-correlation between data points leading to breakdown of the backprop algorithm. To address this challenge, we present an asynchronous variational inference method that allows efficient end-to-end training. Our method achieves a classification mAP of 22.4% on the Charades [42] benchmark, outperforming the state-of-the-art (17.2% mAP), and offers equal gains on the task of temporal localization.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr.2017.599

Cites

19 of 19 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 47

  1. schwing, 2015, Fully Connected Deep Structured Networks
  2. salakhutdinov, 2009, Deep boltzmann machines, AISTATS
  3. 10.21236/ADA612444 · DOI
  4. 10.1109/CVPR.2014.85 · DOI · Paper page
  5. 10.1109/CVPR.2012.6247806 · DOI · Paper page
  6. 10.1109/CVPR.2007.383487 · DOI · Paper page
  7. rohrbach, 2012, Script data for attribute-based recognition of composite activities, ECCV
  8. 10.1109/TPAMI.2011.158 · DOI · Paper page
  9. weinzaepfel, 2016, Towards weakly-supervised action localization
  10. 10.1109/CVPR.2016.597 · DOI · Paper page
  11. 10.1109/CVPR.2015.7298789 · DOI · Paper page
  12. yeung, 2015, Every moment counts Dense detailed labeling of actions in complex videos
  13. mettes, 2016, Spot on: Action localization from pointly-supervised proposals, ECCV
  14. 10.1109/CVPR.2015.7299101 · DOI · Paper page
  15. zhang, 2016, Instance-level segmentation with deep densely connected mrfs, CVPR
  16. peng, 2014, Action recognition with stacked fisher vectors, ECCV
  17. dalal, 2006, Human detection using oriented histograms of flow and appearance, ECCV
  18. chen, 2015, Learning Deep Structured Models, Proc ICML
  19. 10.1109/ICCV.2015.517 · DOI · Paper page
  20. 10.1007/s11263-005-1838-7 · DOI · Paper page
  21. lan, 2015, Beyond gaussian pyramid: Multi-skip feature stacking for action recognition, CVPR
  22. 10.1109/CVPR.2011.5995496 · DOI · Paper page
  23. 10.1109/CVPR.2008.4587756 · DOI · Paper page
  24. 10.1109/ICCVW.2009.5457659 · DOI · Paper page
  25. 10.1109/ICCV.2007.4408872 · DOI · Paper page
  26. 10.1109/ICCV.2015.522 · DOI · Paper page
  27. 10.1109/CVPR.2012.6247808 · DOI · Paper page
  28. 10.1016/j.cviu.2010.10.002 · DOI · Paper page
  29. 10.1109/CVPR.2016.291 · DOI · Paper page
  30. 10.1109/CVPR.2016.18 · DOI · Paper page
  31. 10.1109/CVPR.2016.327 · DOI
  32. 10.1007/978-3-642-15567-3_11 · DOI · Paper page
  33. gupta, 2015, Visual semantic role labeling
  34. 10.1109/CVPR.2016.119 · DOI · Paper page
  35. 10.1007/978-3-642-33765-9_31 · DOI
  36. 10.1109/CVPR.2013.332 · DOI
  37. 10.5244/C.22.99 · DOI · Paper page
  38. de souza, 2016, Sympathy for the details: Dense trajectories and hybrid classification architectures for action recognition, ECCV
  39. 10.1109/CVPR.2015.7299176 · DOI · Paper page
  40. gorban, 2015, THUMOS Challenge Action Recognition with A Large Number of Classes
  41. 10.1109/CVPR.2015.7298676 · DOI · Paper page
  42. 10.1109/ICCV.2013.453 · DOI · Paper page
  43. 10.1109/TPAMI.2009.83 · DOI · Paper page
  44. 10.1109/CVPR.2013.457 · DOI · Paper page
  45. simonyan, 2015, Very deep convolutional networks for large-scale image recognition, ICLRE
  46. sigurdsson, 2016, Hollywood in homes: Crowdsourcing data collection for activity understanding, ECCV
  47. 10.1007/978-3-319-46454-1_5 · DOI

Cited by

See all 144

Metadata

Authors
Sigurdsson, G. A., Divvala, S., Farhadi, A., & Gupta, A.
Venue
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2017-07
Pages
5650-5659
ISSN
1063-6919 (ISSN-L)
OpenAlex
W2583815496
Cited by
144
References
19
FWCI
6.63
Type
conference paper
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.