The DOI matches its Crossref record; 44 of the 46 references it cites were found in a registry or an index.
Two-Stream 3-D convNet Fusion for Action Recognition in Videos With Arbitrary Size and Length
Abstract
3-D convolutional neural networks (3-D-convNets) have been very recently proposed for action recognition in videos, and promising results are achieved. However, existing 3-D-convNets has two “artificial” requirements that may reduce the quality of video analysis: 1) It requires a fixed-sized (e.g., 112 $\times$ 112) input video; and 2) most of the 3-D-convNets require a fixed-length input (i.e., video shots with fixed number of frames). To tackle these issues, we propose an end-to-end pipeline named Two-stream 3-D-convNet Fusion, which can recognize human actions in videos of arbitrary size and length using multiple features. Specifically, we decompose a video into spatial and temporal shots. By taking a sequence of shots as input, each stream is implemented using a spatial temporal pyramid pooling (STPP) convNet with a long short-term memory (LSTM) or CNN-E model, softmax scores of which are combined by a late fusion. We devise the STPP convNet to extract equal-dimensional descriptions for each variable-size shot, and we adopt the LSTM/CNN-E model to learn a global description for the input video using these time-varying descriptions. With these advantages, our method should improve all 3-D CNN-based video analysis methods. We empirically evaluate our method for action recognition in videos and the experimental results show that our method outperforms the state-of-the-art methods (both 2-D and 3-D based) on three standard benchmark datasets (UCF101, HMDB51 and ACT datasets).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It is also worth pointing out that DH [35], another deep-based hashing method in those baselines, yields unsatisfactory performance on all datasets, where three speculations are illustrated for those results as follows.”
From Unsupervised Deep Video Hashing via Balanced Code for Large-Scale Video Retrieval (2019) · cites this paper as [35]
Citations
Cites
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,313
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Gool, L. V. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_2 · Cited by 4,232
A Convolutional Neural Network for Modelling Sentences
Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014) · Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.3115/v1/p14-1062 · Cited by 3,601
Learning realistic human actions from movies
Ivan Laptev, A., Marszałek, M., Schmid, C., & Rozenfeld, B. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587756 · Cited by 3,481
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
Cited by
Sorted by citations
Items 101–200 of 263
Human Action Recognition by Discriminative Feature Pooling and Video Segment Attention Model
Moniruzzaman, M., Yin, Z., He, Z., Qin, R., & Leu, M. C. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3058050 · Cited by 22
Binary dense sift flow based two stream CNN for human action recognition
Park, S. K., Chung, J. H., Kang, T., & Lim, M. T. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-10795-2 · Cited by 22
Ou, D., Tan, K., Du, Q., Zhu, J., Wang, X., & Chen, Y. (2019) · Remote Sensing · doi:10.3390/rs11060654 · Cited by 21
“Despite the fact that deep learning based methods have now been developed for HSI classification, including convolutional neural networks (CNNs) [6–8], 3D convolutional neural networks (3D-CNNs) [9,10], and long short-term memory (LSTM) networks [11,12], these problems still exist.”
BackgroundSelf-citationExploiting Mid-Level Semantics for Large-Scale Complex Video Classification
Zhang, J., Mei, K., Zheng, Y., & Jianping Fan, B. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2907453 · Cited by 22
“addition to CNNs, researchers have employed LSTMs to model the temporal dynamics in videos [29], [31]–[34].”
MethodWeGAN: Deep Image Hashing With Weighted Generative Adversarial Networks
Wang, Y., Zhang, L., Nie, F., Li, X., Chen, Z., & Wang, F. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2947197 · Cited by 21
“Recently deep learning has attracted increasing interest in the fields of computer vision and machine learning, and a variety of deep learning algorithms have been proposed over the past few years [31], [32], [40]–[42].”
BackgroundSpatial-temporal saliency action mask attention network for action recognition
Jiang, M., Pan, N., & Kong, J. (2020) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2020.102846 · Cited by 21
A Novel Deep Learning Based Automated Academic Activities Recognition in Cyber-Physical Systems
Wasim, M., Ahmed, I., Ahmad, J., & Hassan, M. M. (2021) · IEEE Access · doi:10.1109/access.2021.3073890 · Cited by 21
“Wang et al. [33] overcome this limitation by proposing another technique that works for arbitrary video length and size.”
Temporal Action Localization Using Long Short-Term Dependency
Zhou, Y., Wang, R., Li, H., & Kung, S. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3042077 · Cited by 22
“tent analysis and has been extensively studied over the last few years [16], [17], [18], [19], [20], [21].”
BackgroundSpatio-Temporal Attention and Gaussian Processes for Personalized Video Gaze Estimation
Jindal, S., Yadav, M., & Manduchi, R. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00065 · Cited by 22
“Similarly, 3D convolutional neural networks (CNNs) can also capture spatiotemporal information from videos, but they require many model parameters [3, 14, 23, 29, 59, 67].”
BackgroundCollaborative Static and Dynamic Vision-Language Streams for Spatio-Temporal Video Grounding
Lin, Z., Tan, C., Hu, J., Jin, Z. J., Ye, T., & Zheng, W. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02212 · Cited by 20
“Considering that static and dynamic visual cues are quite different but complementary in video understanding, a number of two-stream models [4,5,21,25, 26, 28] are proposed to capture these two kinds of visual cues.”
BackgroundSoltanian, M. R. K., & Ghaemmaghami, S. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2844101 · Cited by 20
“Static frame-based CNN descriptors [1], [33], [34], [39], [40], and spatio-temporal CNN descriptors [41]–[44] are two main approaches that employ CNNs in event detection.”
Structure-Constrained Motion Sequence Generation
Yan, Y., Ni, B., Zhang, W., Xu, J., & Yang, X. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2885235 · Cited by 20
“important roles in solving different problems, including human motion tracking [38]–[41], action recognition [42]–[46], and pose estimation [47]–[49].”
BackgroundSpatiotemporal Symmetric Convolutional Neural Network for Video Bit-Depth Enhancement
Liu, J., Liu, P., Su, Y., Jing, P., & Yang, X. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2897909 · Cited by 20
“CNNs have been widely applied in a tremendous number of video processing tasks, including action recognition [14], tracking [15], and segmentation [16].”
BackgroundProtecting multi-party privacy in location-aware social point-of-interest recommendation
Wang, W., Liu, A., Li, Z., Zhang, X., Li, Q., & Zhou, X. (2018) · World Wide Web · doi:10.1007/s11280-018-0550-9 · Cited by 19
“With rich data and advanced data processing methods [9,37], more accurate and personalized POI recommendation can be achieved.”
MethodData-Driven Approaches for Spatio-Temporal Analysis: A Survey of the State-of-the-Arts
Das, M., & Ghosh, S. K. (2020) · Journal of Computer Science and Technology · doi:10.1007/s11390-020-9349-0 · Cited by 19
The Devil is in the Details: An Efficient Convolutional Neural Network for Transport Mode Detection
Moreau, H., Vassilev, A., & Chen, L. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2021.3110949 · Cited by 19
“By replacing the flatten step with a global pooling operation [28], we obtain a vector which is as long as the input tensor had feature maps, no matter how many time points the input had (see fig.”
MethodUni-and-Bi-Directional Video Prediction via Learning Object-Centric Transformation
Chen, X., & Wang, W. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2946475 · Cited by 18
“particular, researchers have made remarkable progress for problems such as action recognition [1], [2], video segmentation [3], and video caption [4].”
BackgroundSu, H., Zhao, X., Lin, T., Liu, S., & Hu, Z. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2999184 · Cited by 18
Multi-Level Temporal Dilated Dense Prediction for Action Recognition
Wang, J., Lin, Y., Zhang, M., Gao, Y., & Ma, A. J. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3087023 · Cited by 18
Hõrak, H. (2019) · Information · doi:10.3390/info10090269 · Cited by 18
“This fusion can be applied in the convolutional layers (early fusion) [90] or the fully connected layers (late fusion) [91,92], either approach can provide task-specific feature learning benefits.”
BackgroundScale-adaptive local binary pattern for texture classification
Pan, Z., Wu, X., & Li, Z. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-08205-9 · Cited by 17
Khowaja, S. A., & Lee, S. (2019) · Neural Computing and Applications · doi:10.1007/s00521-019-04578-y · Cited by 17
“Wang et al. [19] also used the 3D ConvNets for extracting the features and performing the fusion before LSTM or convolutional layer embedding (CNN-E) stacking.”
MethodSahoo, S. P., Modalavalasa, S., & Ari, S. (2022) · Digital Signal Processing · doi:10.1016/j.dsp.2022.103763 · Cited by 16
YOWOv3: A Lightweight Spatio-Temporal Joint Network for Video Action Detection
Zhu, A., Wang, Y., Yang, J., Yan, T., Ma, H., & Li, W. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3387933 · Cited by 16
“Another prevalent approach involves directly classifying sequences by feeding CNN features into the LSTM, which is an effective method for deriving spatial-temporal representations of videos [30], [31], [32].”
MethodLow-rank dimensionality reduction for multi-modality neurodegenerative disease identification
Zhu, X., Suk, H., & Shen, D. G. (2018) · World Wide Web · doi:10.1007/s11280-018-0645-3 · Cited by 16
“Alzheimer’s Disease (AD) is an irreversible and progressive brain disorder and typically begins with mild memory loss and later may seriously impair an individual’s ability of daily activities [12, 25, 26, 30, 44].”
BackgroundPlay and rewind: Context-aware video temporal action proposals
Gao, L., Li, T., Song, J., Zhao, Z., & Shen, H. T. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107477 · Cited by 16
Self-citationAppearance-and-Dynamic Learning With Bifurcated Convolution Neural Network for Action Recognition
Zhang, J., Hu, H., & Liu, Z. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3006223 · Cited by 16
Sparse Deep LSTMs with Convolutional Attention for Human Action Recognition
Aghaei, A., Nazari, A., & Moghaddam, M. E. (2021) · SN Computer Science · doi:10.1007/s42979-021-00576-x · Cited by 15
A parameter estimation method of the simple PCNN model for infrared human segmentation
He, F., Guo, Y., & Gao, C. (2018) · Optics & Laser Technology · doi:10.1016/j.optlastec.2018.05.042 · Cited by 16
Geng, Y., Du, J., & Liang, M. (2018) · World Wide Web · doi:10.1007/s11280-018-0603-0 · Cited by 16
“In terms of action recognition, [40] proposed an end-to-end framework called Twostream 3D convNet Fusion, which used multiple features to recognize human actions in videos of arbitrary size and length.”
BackgroundFusion of spatial and dynamic CNN streams for action recognition
Russel, N. S., & Arivazhagan, S. (2021) · Multimedia Systems · doi:10.1007/s00530-021-00773-x · Cited by 16
Action Recognition Using Action Sequences Optimization and Two-Stream 3D Dilated Neural Network
Xiong, X., Min, W., Han, Q., Wang, Q., & Zha, C. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/6608448 · Cited by 15
“Finally, each flow obtains the corresponding class scores before the classification we referred to in [53] to fuse the scores of the two networks.”
BackgroundGuttikonda, J. B., & Sridevi, R. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-7168-5 · Cited by 15
Universal-to-Specific Framework for Complex Action Recognition
Zhao, P., Xie, L., Zhang, Y., & Tian, Q. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3025665 · Cited by 16
Probabilistic Semantic Retrieval for Surveillance Videos With Activity Graphs
Chen, Y., Wang, J., Bai, Y., Castanon, G. D., & Saligrama, V. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2865860 · Cited by 14
“[20] propose a novel LSTM/CNN-E model to learn a global description for the input video using time-varying descriptions extracted from the STPP ConvNet.”
MethodZhang, Y., Guo, Q. J., Du, Z., & Wu, A. (2023) · Electronics · doi:10.3390/electronics12030538 · Cited by 14
Human Action Representation Learning Using an Attention-Driven Residual 3DCNN Network
Ullah, H., & Munir, A. (2023) · Algorithms · doi:10.3390/a16080369 · Cited by 16
“The STPP + LSTM [46] method secured the second-best position, with SPF and FPS values of 0.0063 and 154.6, respectively.”
MethodInfluentialExplore Video Clip Order With Self-Supervised and Curriculum Learning for Video Applications
Xiao, J., Li, L., Xu, D., Long, C., Shao, J., Zhang, S., Pu, S., & Zhuang, Y. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3025661 · Cited by 14
“The 3D CNNs [5], [32], [33] extend 2D CNNs to the temporal domain and extract spatiotemporal features for action recognition.”
BackgroundGlobal-view hashing: harnessing global relations in near-duplicate video retrieval
Jing, W., Nie, X., Cui, C., Xi, X., Yang, G., & Yin, Y. (2018) · World Wide Web · doi:10.1007/s11280-018-0536-7 · Cited by 14
Multi-Scale Based Context-Aware Net for Action Detection
Liu, H., Wang, S., Wang, W., & Cheng, J. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2929923 · Cited by 15
“Action Recognition: Action recognition has been extensively explored in recent years [15], [23], [32], [36], [38], [39], [45],”
BackgroundVideo Activity Recognition With Varying Rhythms
Ayhan, B., Kwan, C., Budavari, B., Larkin, J., Gribben, D., & Li, B. (2020) · IEEE Access · doi:10.1109/access.2020.3033190 · Cited by 16
“As an example in [18], the authors used UCF101 and HMDB51 datasets for demonstrating their two-stream 3-D-convNet fusion pipeline, which can recognize human The associate editor coordinating the review of this manuscript and approving it for publication was Jiachen Yang . actions in videos of…”
MethodRSGNet: Relation based Skeleton Graph Network for Crowded Scenes Pose Estimation
Dai, Y., Wang, X., Gao, L., Song, J., & Shen, H. T. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16206 · Cited by 14
“It has attracted huge interest, since it supports wide applications, such as human parsing (Fang et al. 2018; Wang et al. 2018), human-computer interaction (Shotton et al.”
BackgroundSelf-citationHuman Action Recognition from Various Data Modalities: A ReviewPreprint
Rahmani, H., Bennamoun, M., & Ke, Q. (2021) · doi:10.36227/techrxiv.13708270.v1 · Cited by 14
Bidirectional LSTM with saliency-aware 3D-CNN features for human action recognition
Arif, S., Wang, J., Siddiqui, A. A., Hussain, R., & Hussain, F. (2021) · Journal of Engineering Research · doi:10.36909/jer.v9i3a.8383 · Cited by 13
A 3DCNN-Based Knowledge Distillation Framework for Human Activity Recognition
Ullah, H., & Munir, A. (2023) · Journal of Imaging · doi:10.3390/jimaging9040082 · Cited by 14
“From the scaled results in Table 16, it can be noticed that the STPP + LSTM [53] method has the best SPF and FPS values of 0.”
BackgroundMethodInfluentialA hybrid model of convolutional neural networks and deep regression forests for crowd counting
Ji, Q., Zhu, T., & Bao, D. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01688-2 · Cited by 13
Soltanian, M. R. K., Amini, S., & Ghaemmaghami, S. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2959426 · Cited by 13
“Static frame-level CNN descriptors [5], [27], and spatio-temporal CNN descriptors [28], [29], are two commonly used CNN based methods to describe visual events.”
MethodKTN: Knowledge Transfer Network for Multi-person DensePose Estimation
Wang, X., Gao, L., Song, J., & Shen, H. T. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3414014 · Cited by 13
“Inspired by successful works on human-centric recognition tasks, involving pedestrian detection [32, 39], pedestrian retrieval [24, 25, 30], human semantics recognition [2, 4, 12, 37, 40], human keypoints detection [7, 11, 23, 28, 35] and human action recognition [9, 14, 31], the two-stage strategy has become the domi…”
BackgroundSelf-citationSemantic-aware Transfer with Instance-adaptive Parsing for Crowded Scenes Pose Estimation
Wang, X., Gao, L., Dai, Y., Zhou, Y., & Song, J. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475233 · Cited by 13
“Notably, the one-hot vector is a global identification of an image, which is semantically strong and has been widely used for general classification problems, such as object recognition [9, 24], short-term action recognition [2, 12, 25, 30] and language (a) Two Patterns of Keypoints Classification Spatially precise”
BackgroundSelf-citationLow-rank hypergraph feature selection for multi-output regression
Zhu, X., Hu, R., Lei, C., Thung, K., Zheng, W., & Wang, C. (2017) · World Wide Web · doi:10.1007/s11280-017-0514-5 · Cited by 13
“By eliminating the global linear assumption in the process of constructing model, Sammon [21] proposed a nonlinear mapping, which preserves the correlation between the samples; Hastie [59] proposed the principal curves, which is defined as a “self-consistent” smooth curve through the probability distribution or data c…”
MethodEstimation of gait normality index based on point clouds through deep auto-encoder
Nguyen, T.-N., & Meunier, J. (2019) · EURASIP Journal on Image and Video Processing · doi:10.1186/s13640-019-0466-z · Cited by 13
Natural Language Generation Using Dependency Tree Decoding for Spoken Dialog Systems
Park, Y., & Kang, S. (2018) · IEEE Access · doi:10.1109/access.2018.2889556 · Cited by 13
“Specifically, the recurrent neural network (RNN) has demonstrated unequivocal advantages in tasks such as machine translation [3], [4], image captioning [5], [6], video captioning [7]–[9], video action recognition [10], [11] video retrieval [12], information extraction [13], [14], and NLG [15], [16].”
BackgroundDilated-aware discriminative correlation filter for visual tracking
Xu, G., Zhu, H., Deng, L., Han, L., Li, Y., & Lu, H. (2018) · World Wide Web · doi:10.1007/s11280-018-0555-4 · Cited by 12
Moniruzzaman, M., & Yin, Z. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3263965 · Cited by 12
“Different from action recognition [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], the temporal action localization task aims to localize action instances withtemporalboundaries.”
BackgroundOne-shot Video Graph Generation for Explainable Action Reasoning
Han, Y., Zhuo, T., Zhang, P., Huang, W., Zha, Y., Zhang, Y., & Kankanhalli, M. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.02.069 · Cited by 12
LgNet: A Local-Global Network for Action Recognition and Beyond
Zhou, J., Fu, Z., Huang, Q., Liu, Q., & Wang, Y. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3189253 · Cited by 12
“Two-stream based methods [8], [13], [17], [18], [31] model appearances and motions separately with two networks and fuse features of the two streams.”
MethodMARES: multitask learning algorithm for Web-scale real-time event summarization
Yang, M., Tu, W., Qu, Q., Lei, K., Chen, X., Zhu, J., & Shen, Y. (2018) · World Wide Web · doi:10.1007/s11280-018-0597-7 · Cited by 12
“Inspired by the successes of deep neural networks in computer vision [14, 43, 49], natural language processing [7], and multimedia [15, 29, 38–40], several recent studies have been proposed to employ deep neural networks to summarization.”
BackgroundTwo-stage deep learning for supervised cross-modal retrieval
Shao, J., Zhao, Z., & Su, F. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-7068-0 · Cited by 12
Future video frame prediction based on generative motion-assistant discriminative network
Li, C., & Chen, X. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110028 · Cited by 11
Correlation based feature fusion for the temporal video scene segmentation task
Kishi, R. M., Trojahn, T. H., & Goularte, R. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6959-4 · Cited by 12
SHORT: Segmented histogram technique for robust real-time object recognition
Bonny, T., Rabie, T. F., Baziyad, M. S., & Balid, W. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-07826-4 · Cited by 11
Nguyen, T.-B., Le, T., Devillaine, L., Pham, T. T. T., & Ngoc, N. P. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-08183-y · Cited by 10
“A variant of RNN structure, LSTM (Long-short Term Memory) that is proven in numerous applications such as video hashing [47], video captioning [46], action recognition [56] is employed to build a final signature for a given individual [40, 60].”
Automatic image annotation via category labels
Zhang, W., Hu, H., Hu, H., & Yu, J. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-019-07929-y · Cited by 10
Searching for Two-Stream Models in Multivariate Space for Video Recognition
Gong, X., Wang, H., Shou, Z., Feiszli, M., Wang, Z., & Yan, Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00793 · Cited by 10
“Early two-stream models [38, 37] do not fuse features from individual streams in the middle of the model.”
BackgroundTemporal-Spatial Neural Filter: Direction Informed End-to-End Multi-channel Target Speech SeparationPreprint
Gu, R., & Zou, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.00391 · Cited by 10
Complex Human Action Recognition in Live Videos Using Hybrid FR-DL MethodPreprint
Rezaei, M., & Serpush, F. (2020) · doi:10.31219/osf.io/fsvz2 · Cited by 10
“According to research conducted in [12], [45], [52], [37], [53], LSTM is capable of learning long-term dependencies, and its special structure includes inputs, outputs and forget gates, which controls long-term sequence recognition.”
BackgroundJoints-Centered Spatial-Temporal Features Fused Skeleton Convolution Network for Action Recognition
Song, W., Chu, T., Li, S., Li, N., Hao, A., & Qin, H. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3324835 · Cited by 11
Toward Making Unsupervised Graph Hashing Discriminative
Ma, C., Gong, C., Li, X., Huang, X., Liu, W., & Yang, J. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2931808 · Cited by 9
Dual-Branch Network With a Subtle Motion Detector for Microaction Recognition in Videos
Mi, Y., Zhang, X., Li, Z., & Wang, S. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2989864 · Cited by 9
“MultiNet) [17], temporal segment network (TSN) [4], two-stream 3D network [66] (Two-Stream 3D Fusion), and two-stream residual networks with blocks of 2D spatial convolution plus 1D temporal convolution (R(2+1)D-Two Stream) [13].”
BackgroundA Hierarchical deep model for food classification from photographs
Yang, H., Kang, S., Park, C., Lee, J.-W., Yu, K., & Min, K. (2020) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2020.04.016 · Cited by 9
“To properly capture and caption food on video, Song et al.'s scheme [29] can be employed.”
MethodAMANet: Adaptive Multi-Path Aggregation for Learning Human 2D-3D Correspondences
Wang, X., Guo, Y., Song, J., Gao, L., & Shen, H. T. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3135145 · Cited by 8
“Due to the successful development of deep neural networks [13]–[18], instance-level tasks, such as object detection [19]–[21], object segmentation [22]–[24], and pose estimation [25]–[29], have made great progress, where two-stage strategy is widely used.”
BackgroundSelf-citationConstruction of Sports Training Management Information System Using AI Action Recognition
Cheng, D., Wang, H., & Li, M. (2022) · Scientific Programming · doi:10.1155/2022/8393612 · Cited by 8
Wan, P., Xue, H., Liu, C., Chen, F., Kong, W., & Zhang, D. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3270307 · Cited by 10
“To further exploit inter-frame correlations, the popular two-stream structure considers the static features of each point and motion information among adjacent frames simultaneously [21], [22], [23], [24].”
BackgroundMulti-stage adaptive regression for online activity recognition
Liu, B., Cai, H., Ju, Z., & Liu, H. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.107053 · Cited by 8
A detection method for low-pixel ratio object
Zhang, R., Yin, D., Ding, J., Luo, Y., Liu, W., Yuan, M., Zhu, C., & Zhou, Z. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6653-6 · Cited by 8
“Nowadays, with convolutional neural network (CNN) rapidly developing, many methods are applied in different fields like face recognition [31] and video analysis [15, 37].”
MethodAuto-X3D: Ultra-Efficient Video Understanding via Finer-Grained Neural Architecture Search
Jiang, Y., Gong, X., Wu, J., Shi, H., Yan, Z., & Wang, Z. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00241 · Cited by 7
“As the name implies, Two-Stream Network [38] uses two streams of image architectures to process single-frame and multi-frame optical flow inputs, respectively.”
InfluentialNighttime Traffic Sign and Pedestrian Detection Using RefineDet with Time‐Series Information
Yamamoto, M., Sultana, R., & Ohashi, G. (2022) · IEEJ Transactions on Electrical and Electronic Engineering · doi:10.1002/tee.23737 · Cited by 7
“There are several ways to use time-series information for a moving scene, such as optical flow [44,45], 3D convolution [46,47], and RNN [16,48,49] Optical flow is obtained from two sequential frames.”
MethodCo-regularized kernel ensemble regression
Wornyo, D. K., Shen, X., Dong, Y. Q., Wang, L., & Huang, S. (2018) · World Wide Web · doi:10.1007/s11280-018-0576-z · Cited by 7
Neelima, N., & Kumar, Y. R. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-07907-4 · Cited by 7
Quadruplet-based deep hashing for image retrieval
Zhu, J., Chen, Z., Li, Z., & Wu, S. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.07.082 · Cited by 7
Lifelog Image Retrieval Based on Semantic Relevance Mapping
Xu, Q., del Molino, A. G., Lin, J., Fang, F., Subbaraju, V., Li, L., & Lim, J. (2021) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3446209 · Cited by 7
“Although recent work on video analytics considers the temporal constraints [45, 50], they deal with data with hightemporal frequency, whereas the lifelog data typically has low-temporal frequency.”
BackgroundZhang, T., & Duan, C. (2021) · IEEE Access · doi:10.1109/access.2021.3128357 · Cited by 7
“And, the output features are fused to be fed into the ECOC-SVM classifier for getting the final classification results [29], [30].”
MethodHigh Efficient 3D Convolution Feature Compression
Cai, Y., Xing, P., & Gao, X. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3200698 · Cited by 7
“Great progress has been made in action recognition based on three-dimensional convolutional neural network (3D CNN) [11], [12], [13], [14], [15], [16].”
Robust visual tracking based on convolutional neural network with extreme learning machine
Sun, R., Wang, X., & Yan, X. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6491-6 · Cited by 7
Self-citationMallick, A. K., & Mukhopadhyay, S. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09312-8 · Cited by 7
“These techniques are used for video captioning [9] and action recognition [35].”
MethodA survey on description and modeling of audiovisual documents
Fourati, M., Jedidi, A., & Gargouri, F. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09589-9 · Cited by 7
Lin, C.-W., Lin, M., & Yang, S. (2020) · IEEE Access · doi:10.1109/access.2020.3032430 · Cited by 6
Time series activity classification using gated recurrent units
Tan, Y.-F., Guo, X., & Poh, S.-C. (2021) · International Journal of Power Electronics and Drive Systems/International Journal of Electrical and Computer Engineering · doi:10.11591/ijece.v11i4.pp3551-3558 · Cited by 6
“The deep learning model that is commonly used in vision-based activity classification is a combination of convolutional neural network (CNN) and recurrent neural network (RNN) [5, 6].”
MethodStudy of Human Motion Recognition Algorithm Based on Multichannel 3D Convolutional Neural Network
Ju, Y. (2021) · Complexity · doi:10.1155/2021/7646813 · Cited by 6
“Two-stream (TS) [23] recognizes actions by constructing a two-stream spatiotemporal network model, but the network is relatively shallow.”
BackgroundYao, L., Yang, W., Huang, W., Jiang, N., & Zhou, B. (2021) · International Journal of Intelligent Systems · doi:10.1002/int.22713 · Cited by 7
“Compared with 2D ‐ CNN methods, 3D ‐ CNN ‐ based meth-ods, 22,23 provide an easier way to handle temporal information.”
MethodAn Alphapose-Based Pedestrian Fall Detection Algorithm
Zhao, X., Hou, F., Su, J., & Davis, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-06794-5_52 · Cited by 6
A supervised learning to index model for approximate nearest neighbor image retrieval
Kan, S., Cen, L., Zheng, X., Cen, Y., Zhu, Z., & Wang, H. (2019) · Signal Processing Image Communication · doi:10.1016/j.image.2019.08.009 · Cited by 6
Sun, P., Yang, Z., Zhang, T., Guo, S., & Chen, F. (2021) · Applied Intelligence · doi:10.1007/s10489-021-02527-8 · Cited by 6
“Two streams [29] are used to extract the dynamics-relevant features from the optical flow data and the global static features from the RGB data, respectively.”
MethodPerceptual hashing method for video content authentication with maximized robustness
Ma, Q., & Xing, L. (2021) · EURASIP Journal on Image and Video Processing · doi:10.1186/s13640-021-00577-z · Cited by 6
“However, these methods are mainly used for the video retrieval, video captioning and visual recognition [21–23].”
MethodGroup sparse based locality – sensitive dictionary learning for video semantic analysis
Benuwa, B., Zhan, Y., Liu, J., Gou, J., Ghansah, B., & Ansah, E. K. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-6417-3 · Cited by 6
“Currently deep learning techniques for video analysis have been widely utilized and has achieve massive successes with feature leaning in its application areas as discussed in [9, 11, 27, 33, 34] for segmentation, annotation, retrieval and captioning of videos.”
MethodPredictively Encoded Graph Convolutional Network for Noise-Robust Skeleton-based Action RecognitionPreprint
Yu, J., Yoon, Y., & Jeon, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.07514 · Cited by 6
Effective shortest travel-time path caching and estimating for location-based services
Zhang, D., Liu, A., Li, Z., Jia, G., Chen, F., & Li, Q. (2018) · World Wide Web · doi:10.1007/s11280-018-0549-2 · Cited by 5
“In our future work, we will try to utilize parallel (like Spark), distributed (like [18]), hashing (like [28, 46]), binary coding (like [27, 45]), and even artificial intelligence (like [10, 11, 26, 33]) processing techniques, to further accelerate path query processing and reduce user query response time.”
MethodDeep Loss Driven Multi-Scale Hashing Based on Pyramid Connected Network
Gu, L., Liu, J., Liu, X., & Sun, J. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2991513 · Cited by 5
“Deep learning has driven great attention in large-scale visual applications, such as image classification [50], image semantics [51], [52], object detection [53], semantic segmentation [54], action recognition [55]–[57], and deep hashing methods [11], [12], [30]–[36] proposed recently.”
BackgroundSelf-citationPhotographic painting style transfer using convolutional neural networks
Khan, A., Ahmad, M. I., Naqvi, N., Yousafzai, F., & Xiao, J. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-7270-8 · Cited by 5
“Wang et al. [47, 48] developed an algorithm for human activity recognition in videos with CNN features which has received increasing attention in multimedia understanding.”
BackgroundGroup sparse reduced rank regression for neuroimaging genetic study
Zhu, X., Suk, H., & Shen, D. G. (2018) · World Wide Web · doi:10.1007/s11280-018-0637-3 · Cited by 5
“, an 2,1-norm regularizer [15, 33, 39, 43, 48, 49]) on each reduced rank matrix, such that highly informative phenotypes and genotypes are selected for the BW-CGA study [38].”
Metadata
- Authors
- Wang, X., Gao, L., Wang, P., Sun, X., & Liu, X.
- Affiliations
- University of Electronic Science and Technology of China (CN)
- The University of Adelaide (AU)
- Harbin Institute of Technology (CN)
- Beihang University (CN)
- Venue
- IEEE Transactions on Multimedia
- Published
- 2018-03
- Volume
- 20
- Issue
- 3
- Pages
- 634-644
- ISSN
- 1520-9210 (ISSN-L)
- 1941-0077
- OpenAlex
- W2751445731
- Cited by
- 263
- References
- 46
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 12 · 2023: 24 · 2022: 23 · 2021: 44 · 2020: 47 · 2019: 60 · 2018: 42
- FWCI
- 17.03
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Multimedia (publishedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- Funders
- Fundamental Research Funds for the Central Universities (ZYGX2014J063, ZYGX2014Z007)
- National Natural Science Foundation of China (61502080, 61632007)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- 3D convolutional neural networks, long short-term memory, decision fusion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
