The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognition
Open accessAbstract
Motion representation plays a vital role in human action recognition in videos. In this study, we introduce a novel compact motion representation for video action recognition, named Optical Flow guided Feature (OFF), which enables the network to distill temporal information through a fast and robust approach. The OFF is derived from the definition of optical flow and is orthogonal to the optical flow. The derivation also provides theoretical support for using the difference between two frames. By directly calculating pixel-wise spatio-temporal gradients of the deep feature maps, the OFF could be embedded in any existing CNN based video action recognition framework with only a slight additional cost. It enables the CNN to extract spatiotemporal information, especially the temporal information between frames simultaneously. This simple but powerful idea is validated by experimental results. The network with OFF fed only by RGB inputs achieves a competitive accuracy of 93.3% on UCF-101, which is comparable with the result obtained by two streams (RGB and optical flow), but is 15 times faster in speed. Experimental results also show that OFF is complementary to other motion modalities such as optical flow. When the proposed method is plugged into the state-of-the-art video action recognition framework, it has 96.0% and 74.2% accuracy on UCF-101 and HMDB-51 respectively. The code for this project is available at: https://github.com/kevin-ssy/Optical-Flow-Guided-Feature
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Horn, B. K. P., & Schunck, B. G. (1981) · Artificial Intelligence · doi:10.1016/0004-3702(81)90024-2 · Cited by 10,312
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Horn, B. K., & Schunck, B. G. (1981) · SPIE Proceedings · doi:10.1117/12.965761 · Cited by 7,394
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Stacked Hourglass Networks for Human Pose Estimation
Newell, A., Yang, K., & Deng, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_29 · Cited by 5,634
Ucf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
Performance of optical flow techniques
Barron, J. L., Fleet, D. J., & Beauchemin, S. S. (1994) · International Journal of Computer Vision · doi:10.1007/bf01420984 · Cited by 4,305
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Van Gool, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_2 · Cited by 3,958
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 3,752
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
Wei, S.-E., Ramakrishna, V., Kanade, T., & Sheikh, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.511 · Cited by 2,966
High Accuracy Optical Flow Estimation Based on a Theory for Warping
Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24673-2_3 · Cited by 2,570
Action recognition by dense trajectories
Wang, H., Klaser, A., Schmid, C., & Liu, C.-L. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995407 · Cited by 2,154
Beyond short snippets: Deep networks for video classification
Ng, J. Y.-H., Hausknecht, M., Vijayanarasimhan, S., Vinyals, O., Monga, R., & Toderici, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299101 · Cited by 2,062
A Spatio-Temporal Descriptor Based on 3D-Gradients
Klaeser, A., Marszalek, M., & Schmid, C. (2008) · Procedings of the British Machine Vision Conference 2008 · doi:10.5244/c.22.99 · Cited by 1,846
A Duality Based Approach for Realtime TV-L 1 Optical Flow
Zach, C., Pock, T., & Bischof, H. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-74936-3_22 · Cited by 1,658
Human Detection Using Oriented Histograms of Flow and Appearance
Dalal, N., Triggs, B., & Schmid, C. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744047_33 · Cited by 1,652
A 3-dimensional sift descriptor and its application to action recognition
Scovanner, P., Ali, S., & Shah, M. (2007) · Proceedings of the 15th ACM international conference on Multimedia · doi:10.1145/1291233.1291311 · Cited by 1,596
Hypercolumns for object segmentation and fine-grained localization
Hariharan, B., Arbelaez, P., Girshick, R., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298642 · Cited by 1,511
Action recognition with trajectory-pooled deep-convolutional descriptors
Wang, L., Qiao, Y., & Tang, X. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299059 · Cited by 1,194
Long-Term Temporal Convolutions for Action Recognition
Varol, G., Laptev, I., & Schmid, C. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2712608 · Cited by 915
Spindle Net: Person Re-identification with Human Body Region Guided Feature Decomposition and Fusion
Zhao, H., Tian, M., Sun, S., Shao, J., Yan, J., Yi, S., Wang, X., & Tang, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.103 · Cited by 877
Self-citationMulti-context Attention for Human Pose Estimation
Chu, X., Yang, W., Ouyang, W., Ma, C., Yuille, A. L., & Wang, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.601 · Cited by 737
Self-citationSpatiotemporal Multiplier Networks for Video Action Recognition
Feichtenhofer, C., Pinz, A., & Wildes, R. P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.787 · Cited by 679
Bag of visual words and fusion methods for action recognition: Comprehensive study and good practice
Peng, X., Wang, L., Wang, X., & Qiao, Y. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2016.03.013 · Cited by 677
Spatiotemporal Residual Networks for Video Action Recognition
Feichtenhofer, C., Pinz, A. J., & Wildes, R. P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02155 · Cited by 597
Human Action Recognition Using Factorized Spatio-Temporal Convolutional Networks
Sun, L., Jia, K., Yeung, D.-Y., & Shi, B. E. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.522 · Cited by 577
Dynamic Image Networks for Action Recognition
Bilen, H., Fernando, B., Gavves, E., Vedaldi, A., & Gould, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.331 · Cited by 571
Multidimensional orientation estimation with applications to texture analysis and optical flow
Bigun, J., Granlund, G. H., & Wiklund, J. (1991) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.85668 · Cited by 560
Learning Feature Pyramids for Human Pose Estimation
Yang, W., Li, S., Ouyang, W., Li, H., & Wang, X. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.144 · Cited by 536
Self-citationReal-Time Action Recognition with Enhanced Motion Vector CNNs
Zhang, B., Wang, L., Wang, Z., Qiao, Y., & Wang, H. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.297 · Cited by 433
Towards Good Practices for Very Deep Two-Stream ConvNets
Wang, L., Xiong, Y., Wang, Z., & Qiao, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.02159 · Cited by 340
ConvNet Architecture Search for Spatiotemporal Feature Learning
Tran, D., Ray, J., Shou, Z., Chang, S., & Paluri, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.05038 · Cited by 315
Rank Pooling for Action Recognition
Fernando, B., Gavves, E., Oramas M., J., Ghodrati, A., & Tuytelaars, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2558148 · Cited by 304
Spatiotemporal Pyramid Network for Video Action Recognition
Wang, Y., Long, M., Wang, J., & Yu, P. S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.226 · Cited by 272
A Key Volume Mining Deep Framework for Action Recognition
Zhu, W., Hu, J., Sun, G., Cao, X., & Qiao, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.219 · Cited by 244
Deep Temporal Linear Encoding Networks
Diba, A., Sharma, V., & Van Gool, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.168 · Cited by 232
Lattice Long Short-Term Memory for Human Action Recognition
Sun, L., Jia, K., Chen, K., Yeung, D. Y., Shi, B. E., & Savarese, S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.236 · Cited by 199
Crafting GBD-Net for Object Detection
Zeng, X., Ouyang, W., Yan, J., Li, H., Xiao, T., Wang, K., Liu, Y., Zhou, Y., Yang, B., Wang, Z., et al. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2745563 · Cited by 142
Self-citationLong-term Temporal Convolutions for Action RecognitionPreprint
Varol, G., Ivan Laptev, A., & Schmid, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.04494 · Cited by 122
ActionFlowNet: Learning Motion Representation for Action Recognition
Ng, J. Y.-H., Choi, J., Neumann, J., & Davis, L. S. (2018) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2018.00179 · Cited by 111
Chained Cascade Network for Object Detection
Ouyang, W., Wang, K., Zhu, X., & Wang, X. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.214 · Cited by 84
Self-citationLearning Long-Term Dependencies for Action Recognition with a Biologically-Inspired Deep Network
Shi, Y., Tian, Y., Wang, Y., Zeng, W., & Huang, T. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.84 · Cited by 72
Spatio-Temporal Vector of Locally Max Pooled Features for Action Recognition in Videos
Duţă, I. C., Ionescu, B. E., Aizawa, K., & Sebe, N. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.341 · Cited by 71
Learning Mutual Visibility Relationship for Pedestrian Detection with a Deep Model
Ouyang, W., Zeng, X., & Wang, X. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-016-0890-9 · Cited by 52
Self-citation
Cited by
X3D: Expanding Architectures for Efficient Video Recognition
Feichtenhofer, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00028 · Cited by 1,233
Vision-based human activity recognition: a survey
Beddiar, D. R., Nini, B., Sabokrou, M., & Hadid, A. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09004-3 · Cited by 505
STM: SpatioTemporal and Motion Encoding for Action Recognition
Jiang, B., Wang, M., Gan, W., Wu, W., & Yan, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00209 · Cited by 473
FlowFormer: A Transformer Architecture for Optical Flow
Huang, Z., Shi, X., Zhang, C., Wang, Q., Cheung, K. C., Qin, H., Dai, J., & Li, H. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19790-1_40 · Cited by 335
Real-Time Intermediate Flow Estimation for Video Frame Interpolation
Huang, Z., Zhang, T., Heng, W., Shi, B., & Zhou, S. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19781-9_36 · Cited by 326
Metadata
- Authors
- Sun, S., Kuang, Z., Sheng, L., Ouyang, W., & Zhang, W.
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 1390-1399
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2964094092
- Cited by
- 343
- References
- 54
- FWCI
- 14.00
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.