Multimodal Gesture Recognition Based on the ResC3D Network
Abstract
Gesture recognition is an important issue in computer vision. Recognizing gestures with videos remains a challenging task due to the barriers of gesture-irrelevant factors. In this paper, we propose a multimodal gesture recognition method based on a ResC3D network. One key idea is to find a compact and effective representation of video sequences. Therefore, the video enhancement techniques, such as Retinex and median filter are applied to eliminate the illumination variation and noise in the input video, and a weighted frame unification strategy is utilized to sample key frames. Upon these representations, a ResC3D network, which leverages the advantages of both residual and C3D model, is developed to extract features, together with a canonical correlation analysis based fusion scheme for blending features. The performance of our method is evaluated in the Chalearn LAP isolated gesture recognition challenge. It reaches 67.71% accuracy and ranks the 1st place in this challenge.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A multimodal gesture recognition system was developed using the 3D Residual CNN (ResC3D) model trained on an RGB-D dataset, with extracted features combined through a canonical correlation scheme.
“In [98], the authors developed a multimodal gesture recognition system using the 3D Residual CNN (ResC3D) model [99] trained on an RGB-D dataset.”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [98]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077 · see citing works
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593 · see citing works
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756 · see citing works
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354 · see citing works
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,222 · see citing works
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228 · see citing works
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Gool, L. V. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_2 · Cited by 4,164 · see citing works
Land, E. H., & McCann, J. J. (1971) · Journal of the Optical Society of America · doi:10.1364/josa.61.000001 · Cited by 4,135 · see citing works
The recognition of human movement using temporal templates
Bobick, A., & Davis, J. W. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.910878 · Cited by 2,830 · see citing works
High Accuracy Optical Flow Estimation Based on a Theory for Warping
Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24673-2_3 · Cited by 2,634 · see citing works
Real-time human pose recognition in parts from single depth images
Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A. W., Finocchio, M., Blake, A., Cook, M., & Moore, R. (2013) · Communications of the ACM · doi:10.1145/2398356.2398381 · Cited by 2,050 · see citing works
A Spatio-Temporal Descriptor Based on 3D-Gradients
Klaeser, A., Marszałek, M., & Schmid, C. (2008) · British Machine Vision Conference (BMVC) · doi:10.5244/c.22.99 · Cited by 1,842 · see citing works
Vision based hand gesture recognition for human computer interaction: a survey
Rautaray, S. S., & Agrawal, A. (2012) · Artificial Intelligence Review · doi:10.1007/s10462-012-9356-9 · Cited by 1,535 · see citing works
Recognizing human action in time-sequential images using hidden Markov model
Yamato, J., Ohya, J., & Ishii, K. (2003) · Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1992.223161 · Cited by 1,273 · see citing works
Large-scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G. D., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · Cited by 1,266 · see citing works
Free viewpoint action recognition using motion history volumes
Weinland, D., Ronfard, R., & Boyer, E. (2006) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.07.013 · Cited by 906 · see citing works
Molchanov, P. A., Yang, X., Gupta, S., Kim, K., Tyree, S., & Kautz, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.456 · Cited by 698 · see citing works
Max-pooling convolutional neural networks for vision-based hand gesture recognition
Nagi, J., Ducatelle, F., Di, G. A., Ciresan, D. C., Meier, U., Giusti, A., Nagi, F., Schmidhuber, J., & Gambardella, L. M. (2011) · IEEE International Conference on Signal and Image Processing Applications (ICSIPA) · doi:10.1109/icsipa.2011.6144164 · Cited by 667 · see citing works
A new method of feature fusion and its application in image recognition
Sun, Q., Zeng, S.-G., Liu, Y., Heng, P., & De-shen, X. (2005) · Pattern Recognition · doi:10.1016/j.patcog.2004.12.013 · Cited by 517 · see citing works
Hidden Conditional Random Fields for Gesture Recognition
Wang, S. B., Quattoni, A., Morency, L., Demirdjian, D., & Darrell, T. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.132 · Cited by 456 · see citing works
ConvNet Architecture Search for Spatiotemporal Feature LearningPreprint
Tran, D., Ray, J., Zheng, S.-F., Chang, S.-F., & Paluri, M. (2017) · arXiv.org · doi:10.48550/arxiv.1708.05038 · Cited by 421 · see citing works
Haghighat, M., Abdel-Mottaleb, M. S., & Alhalabi, W. (2016) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2016.2569061 · Cited by 380 · see citing works
ChaLearn Looking at People RGB-D Isolated and Continuous Datasets for Gesture Recognition
Wan, J., Li, S. Z., Zhao, Y., Zhou, S., Guyon, I., & Escalera, S. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.100 · Cited by 281 · see citing works
Multimodal Gesture Recognition Using 3-D Convolution and Convolutional LSTM
Zhu, G., Zhang, L., Shen, P., & Song, J. (2017) · IEEE Access · doi:10.1109/access.2017.2684186 · Cited by 269 · see citing works
Real-time fingertip tracking and gesture recognition
Oka, K., Sato, Y., & Koike, H. (2002) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2002.1046630 · Cited by 263 · see citing works
A shape- and texture-based enhanced Fisher classifier for face recognition
Liu, C., & Wechsler, H. (2001) · IEEE Transactions on Image Processing · doi:10.1109/83.913594 · Cited by 261 · see citing works
Gesture modeling and recognition using finite state machines
Hong, P., Turk, M., & Huang, T. S. (2002) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2000.840667 · Cited by 215 · see citing works
Dynamic time warping for off-line recognition of a small gesture vocabulary
Corradini, A. (2002) · Analysis and Modeling of Faces and Gestures · doi:10.1109/ratfg.2001.938914 · Cited by 174 · see citing works
Unsupervised Extraction of Video Highlights via Robust Recurrent Auto-Encoders
Yang, H., Wang, B., Lin, S. C.-F., Wipf, D., Guo, M., & Guo, B. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.526 · Cited by 173 · see citing works
Spatio-temporal covariance descriptors for action and gesture recognition
Sanin, A., Sanderson, C., Harandi, M. T., & Lovell, B. C. (2013) · Proceedings · doi:10.1109/wacv.2013.6475006 · Cited by 154 · see citing works
Wang, P., Li, W., Gao, Z., Zhang, Y., Tang, C., & Ogunbona, P. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.52 · Cited by 151 · see citing works
Spatio-temporal covariance descriptors for action and gesture recognition
Sanin, A., Sanderson, C., Harandi, M. T., & Lovell, B. C. (2013) · Cited by 123 · see citing works
The ChaLearn gesture dataset (CGD 2011)
Guyon, I., Athitsos, V., Jangyodsuk, P., & Escalante, H. J. (2014) · Machine Vision and Applications · doi:10.1007/s00138-014-0596-3 · Cited by 103 · see citing works
Explore Efficient Local Features from RGB-D Data for One-Shot Learning Gesture Recognition
Wan, J., Guo, G., & Li, S. Z. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2513479 · Cited by 102 · see citing works
Large-scale Isolated Gesture Recognition using Convolutional Neural Networks
Wang, P., Li, W., Liu, S., Gao, Z., Tang, C., & Ogunbona, P. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7899599 · Cited by 100 · see citing works
Large-scale gesture recognition with a fusion of RGB-D data based on the C3D model
Li, Y., Miao, Q., Tian, K., Fan, Y., Xu, X., Li, R., & Song, J. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7899602 · Cited by 89 · see citing works
Self-citationVisual understanding of dynamic hand gestures
Yeasin, M., & Chaudhuri, S. (2000) · Pattern Recognition · doi:10.1016/s0031-3203(99)00175-2 · Cited by 88 · see citing works
A Unified Framework for Multi-Modal Isolated Gesture Recognition
Duan, J., Wan, J., Zhou, S., Guo, X., & Li, S. Z. (2018) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3131343 · Cited by 85 · see citing works
Generalized K–L transform based combined feature extraction
Yang, J. J., & Yang, J.-Y. (2002) · Pattern Recognition · doi:10.1016/s0031-3203(01)00152-2 · Cited by 84 · see citing works
Wan, J., Escalera, S., Anbarjafari, G., Escalante, H. J., Baró, X., Guyon, I., Madadi, M., Allïk, J., Gorbova, J., Lin, C., et al. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.377 · Cited by 77 · see citing works
Large-scale Isolated Gesture Recognition using pyramidal 3D convolutional networks
Zhu, G., Zhang, L., Mei, L., Shao, J., Song, J., & Shen, P. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7899601 · Cited by 71 · see citing works
A temporal Bayesian model for classifying, detecting and localizing activities in video sequences
Malgireddy, M. R., Nwogu, I., & Govindaraju, V. (2012) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2012.6239185 · Cited by 51 · see citing works
Wan, J., Ruan, Q.-Q., Li, W., An, G., & Zhao, R. (2014) · J. Electronic Imaging · doi:10.1117/1.jei.23.2.023017 · Cited by 49 · see citing works
Language-Motivated Approaches to Action Recognition
Malgireddy, M. R., Nwogu, I., & Govindaraju, V. (2017) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-57021-1_5 · Cited by 36 · see citing works
Enhancement of Ultrasonic Image Based on the Multi-Scale Retinex Theory
Huang, Y., Gao, Y., Wang, H., Hao, D., Zhao, J., & Zhen, Z. (2012) · Lecture notes in electrical engineering · doi:10.1007/978-3-642-25792-6_18 · Cited by 1 · see citing works
Flagged first, then by citations
3 of the 50 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Bayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A. (2022) · The Visual Computer · doi:10.1007/s00371-021-02166-7 · Cited by 456 · see citing works
MMTM: Multimodal Transfer Module for CNN Fusion
Joze, H. R. V., Shaban, A., Iuzzolino, M. L., & Koishida, K. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01330 · Cited by 323 · see citing works
Improving the Performance of Unimodal Dynamic Hand-Gesture Recognition With Multimodal Training
Abavisani, M., Joze, H. R. V., & Patel, V. M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00126 · Cited by 190 · see citing works
Gesture Recognition: Focus on the Hands
Narayana, P., Beveridge, J. R., & Draper, B. A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00549 · Cited by 152 · see citing works
Motion Fused Frames: Data Level Fusion Strategy for Hand Gesture Recognition
Köpüklü, O., Köse, N., & Rigoll, G. (2018) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00284 · Cited by 141 · see citing works
Searching Multi-Rate and Multi-Modal Temporal Enhanced Networks for Gesture Recognition
Yu, Z., Zhou, B., Wan, J., Wang, P., Chen, H., Liu, X., Li, S. Z., & Zhao, G. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3087348 · Cited by 135 · see citing works
An Efficient PointLSTM for Point Clouds Based Gesture Recognition
Min, Y.-C., Zhang, Y., Chai, X., & Chen, X.-L. (2020) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr42600.2020.00580 · Cited by 100 · see citing works
Elboushaki, A., Hannane, R., Afdel, K., & Koutti, L. (2019) · Expert Systems with Applications · doi:10.1016/j.eswa.2019.112829 · Cited by 134 · see citing works
Dynamic Hand Gesture Recognition Based on 3D Hand Pose Estimation for Human–Robot Interaction
Gao, Q. H., Chen, Y., Ju, Z., & Liang, Y. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3059685 · Cited by 124 · see citing works
Continuous Gesture Segmentation and Recognition Using 3DCNN and Convolutional LSTM
Zhu, G., Zhang, L., Shen, P., Song, J., Shah, S. A. A., & Bennamoun, M. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2869278 · Cited by 117 · see citing works
Uncertainty-Aware Multiview Deep Learning for Internet of Things Applications
Xu, C. X., Zhao, W. Z., Zhao, J., Guan, Z., Song, X., & Li, J. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3206343 · Cited by 117 · see citing works
Attention in convolutional LSTM for gesture recognition
Zhang, L., Zhu, G., Mei, L., Shen, P., Shah, S. A. A., & Bennamoun, M. (2018) · neural information processing systems · Cited by 100 · see citing works
Video-based isolated hand sign language recognition using a deep cascaded model
Rastgoo, R., Kiani, K., & Escalera, S. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09048-5 · Cited by 91 · see citing works
Redundancy and Attention in Convolutional LSTM for Gesture Recognition
Zhu, G., Zhang, L., Yang, L., Mei, L., Shah, S. A. A., Bennamoun, M., & Shen, P. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2919764 · Cited by 90 · see citing works
Basketball action recognition based on the combination of YOLO and a deep fuzzy LSTM network
Khobdeh, S. B., Yamaghani, M. R., & Khodaparast, S. (2023) · The Journal of Supercomputing · doi:10.1007/s11227-023-05611-7 · Cited by 90 · see citing works
Video Hand Gestures Recognition Using Depth Camera and Lightweight CNN
Leon, D. G., Groli, J., Yeduri, S. R., Rossier, D., Mosqueron, R., Pandey, O. J., & Cenkeramaddi, L. R. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3181518 · Cited by 83 · see citing works
RGB-D sensing based human action and interaction analysis: A survey
Liu, B., Cai, H., Ju, Z., & Liu, H. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.05.020 · Cited by 78 · see citing works
Analyzing human–human interactions: A survey
Stergiou, A., & Poppe, R. W. (2019) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2019.102799 · Cited by 80 · see citing works
Fingerspelling Recognition in the Wild With Iterative Visual Attention
Shi, B., Rio, A. M. D., Keane, J., Brentari, D., Shakhnarovich, G., & Livescu, K. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00550 · Cited by 76 · see citing works
Gao, Q., Liu, J., & Ju, Z. (2020) · Expert Systems · doi:10.1111/exsy.12490 · Cited by 63 · see citing works
Online Dynamic Hand Gesture Recognition Including Efficiency Analysis
Köpüklü, O., Gunduz, A., Köse, N., & Rigoll, G. (2020) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2020.2968216 · Cited by 61 · see citing works
Improving Real-Time Hand Gesture Recognition with Semantic Segmentation
Benítez-García, G., Prudente-Tixteco, L., Castro-Madrid, L. C., Toscano-Medina, R., Olivares-Mercado, J., Sánchez-Pérez, G., & Villalba, L. J. G. (2021) · Sensors · doi:10.3390/s21020356 · Cited by 59 · see citing works
Active Exploration of Multimodal Complementarity for Few-Shot Action Recognition
Wanyan, Y., Yang, X., Chen, C., & Xu, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00628 · Cited by 58 · see citing works
An Efficient Federated Multiview Fuzzy C-Means Clustering Method
Hu, X., Qin, J., Shen, Y., Pedrycz, W., Liu, X., & Liu, J. (2023) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2023.3335361 · Cited by 59 · see citing works
SelfGCN: Graph Convolution Network With Self-Attention for Skeleton-Based Action Recognition
Wu, Z., Sun, P., Chen, X., Tang, K., Xu, T., Zou, L., Wang, X., Tan, M., Cheng, F., & Weise, T. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3433581 · Cited by 59 · see citing works
Review of dynamic gesture recognition
Shi, Y., Li, Y., Xiaolong, F. U., Kaibin, M., & Miao, Q. (2021) · Virtual Reality & Intelligent Hardware · doi:10.1016/j.vrih.2021.05.001 · Cited by 58 · see citing works
Self-citationWearable Sensors-Based Hand Gesture Recognition for Human–Robot Collaboration in Construction
Wang, X., Veeramani, D., & Zhu, Z. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3222801 · Cited by 53 · see citing works
Cárdenas, E. J. E., & Cámara-Chávez, G. (2020) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2020.102772 · Cited by 50 · see citing works
How to Design a Three-Stage Architecture for Audio-Visual Active Speaker Detection in the Wild
Köpüklü, O., Taseska, M., & Rigoll, G. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00123 · Cited by 51 · see citing works
Li, C., Zhang, X., Liao, L., Jin, L., & Yang, W. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018585 · Cited by 50 · see citing works
Vision–based framework for automatic interpretation of construction workers' hand gestures
Wang, X., & Zhu, Z. (2021) · Automation in Construction · doi:10.1016/j.autcon.2021.103872 · Cited by 50 · see citing works
Gao, Q. H., Ju, Z., Chen, Y., Wang, Q., & Chi, C. (2022) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2022.3206663 · Cited by 49 · see citing works
Multimodal vision-based human action recognition using deep learning: a review
Shafizadegan, F., Naghsh‐Nilchi, A. R., & Shabaninia, E. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10730-5 · Cited by 55 · see citing works
Analysis of Deep Fusion Strategies for Multi-Modal Gesture Recognition
Roitberg, A., Pollert, T., Haurilet, M., Martín, M., & Stiefelhagen, R. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00029 · Cited by 47 · see citing works
Multi-View Fuzzy Classification With Subspace Clustering and Information Granules
Hu, X., Liu, X., Pedrycz, W., Liao, Q., Shen, Y., Li, Y., & Wang, S. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3231929 · Cited by 40 · see citing works
Self-citationChaLearn Looking at People: IsoGD and ConGD Large-Scale RGB-D Gesture Recognition
Wan, J., Lin, C., Wen, L., Li, Y., Miao, Q., Escalera, S., Anbarjafari, G., Guyon, I., Guo, G., & Li, S. Z. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3012092 · Cited by 37 · see citing works
Self-citationTransfer Learning For Videos: From Action Recognition To Sign Language Recognition
Sarhan, N., & Frintrop, S. (2020) · Proceedings - International Conference on Image Processing · doi:10.1109/icip40778.2020.9191289 · Cited by 37 · see citing works
Multi-deep features fusion for high-resolution remote sensing image scene classification
Yuan, B., Han, L., Gu, X., & Yan, H. (2020) · Neural Computing and Applications · doi:10.1007/s00521-020-05071-7 · Cited by 36 · see citing works
A Hybrid Network for Large-Scale Action Recognition from RGB and Depth Modalities
Wang, H., Song, Z., Li, W., & Wang, P. (2020) · Sensors · doi:10.3390/s20113305 · Cited by 35 · see citing works
Selective spatiotemporal features learning for dynamic gesture recognition
Tang, X., Yan, Z., Peng, J., Hao, B., Wang, H., & Li, J. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114499 · Cited by 35 · see citing works
Multi-Scale Attention 3D Convolutional Network for Multimodal Gesture Recognition
Chen, H., Li, Y., Fang, H., Xin, W., Lu, Z., & Miao, Q. (2022) · Sensors · doi:10.3390/s22062405 · Cited by 35 · see citing works
Self-citationRegional Attention with Architecture-Rebuilt 3D Network for RGB-D Gesture Recognition
Zhou, B., Li, Y., & Wan, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i4.16471 · Cited by 34 · see citing works
Self-citationLin, C., Wan, J., Liang, Y., & Li, S. Z. (2018) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2018.00018 · Cited by 33 · see citing works
Multimodal Hand Gesture Classification for the Human–Car Interaction
D’Eusanio, A., Simoni, A., Pini, S., Borghi, G., Vezzani, R., & Cucchiara, R. (2020) · Informatics · doi:10.3390/informatics7030031 · Cited by 33 · see citing works
LD-ConGR: A Large RGB-D Video Dataset for Long-Distance Continuous Gesture Recognition
Liu, D., Zhang, L., & Wu, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00330 · Cited by 32 · see citing works
A Unified Multimodal De- and Re-Coupling Framework for RGB-D Motion Recognition
Zhou, B., Wang, P., Wan, J., Liang, Y., & Wang, F. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3274783 · Cited by 38 · see citing works
Multimodal Spatiotemporal Networks for Sign Language Recognition
Zhang, S., Meng, W., Li, H., & Cui, X. (2019) · IEEE Access · doi:10.1109/access.2019.2959206 · Cited by 30 · see citing works
Sign language recognition based on global-local attention
Zhang, S., & Zhang, Q. (2021) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2021.103280 · Cited by 30 · see citing works
Correlation Net: Spatiotemporal multimodal deep learning for action recognition
Yudistira, N., & Kurita, T. (2019) · Signal Processing Image Communication · doi:10.1016/j.image.2019.115731 · Cited by 29 · see citing works
Vision-based hand signal recognition in construction: A feasibility study
Wang, X., & Zhu, Z. (2021) · Automation in Construction · doi:10.1016/j.autcon.2021.103625 · Cited by 29 · see citing works
How does hand gestures in videos impact social media engagement - Insights based on deep learning
Anand, K., Urolagin, S., & Mishra, R. K. (2021) · International Journal of Information Management Data Insights · doi:10.1016/j.jjimei.2021.100036 · Cited by 28 · see citing works
Multi-Task and Multi-Modal Learning for RGB Dynamic Gesture Recognition
Fan, D., Lu, H., Xu, S., & Cao, S. C. (2021) · IEEE Sensors Journal · doi:10.1109/jsen.2021.3123443 · Cited by 28 · see citing works
Neural Network Based Lidar Gesture Recognition for Realtime Robot Teleoperation
Chamorro, S., Collier, J. A., & Grondin, F. (2021) · IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR) · doi:10.1109/ssrr53300.2021.9597855 · Cited by 25 · see citing works
Literature review of vision‐based dynamic gesture recognition using deep learning techniques
Jain, R., Karsh, R. K., & Barbhuiya, A. A. (2022) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.7159 · Cited by 25 · see citing works
Convolutional Transformer Fusion Blocks for Multi-Modal Gesture Recognition
Hampiholi, B., Jarvers, C., Mader, W., & Heiko Neumann, B. (2023) · IEEE Access · doi:10.1109/access.2023.3263812 · Cited by 24 · see citing works
Xu, C., Wu, X., Wang, M., Qiu, F., Liu, Y., & Ren, J. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.12.022 · Cited by 22 · see citing works
A Novel Assistive Glove to Convert Arabic Sign Language into Speech
Alzubaidi, M. A., Otoom, M., & Rwaq, A. M. A. (2022) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3545113 · Cited by 21 · see citing works
Dynamic Sign Language Recognition Based on Convolutional Neural Networks and Texture Maps
Cárdenas, E. J. E., Cerna, L. R., & Cámara-Chávez, G. (2019) · SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) · doi:10.1109/sibgrapi.2019.00043 · Cited by 21 · see citing works
Fusing Posture and Position Representations for Point Cloud-Based Hand Gesture Recognition
Bigalke, A., & Heinrich, M. P. (2021) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv53792.2021.00071 · Cited by 20 · see citing works
A spatiotemporal attention-based ResC3D model for large-scale gesture recognition
Li, Y., Miao, Q., Qi, X., Ma, Z., & Ouyang, W. (2018) · Machine Vision and Applications · doi:10.1007/s00138-018-0996-x · Cited by 20 · see citing works
Self-citationFineHand: Learning Hand Shapes for American Sign Language Recognition
Hosain, A. A., Santhalingam, P. S., Pathak, P. H., Rangwala, H., & Košecká, J. (2020) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg47880.2020.00062 · Cited by 20 · see citing works
Chen, G., Dong, Z., Wang, J., & Xia, L. (2022) · Complex & Intelligent Systems · doi:10.1007/s40747-022-00858-8 · Cited by 19 · see citing works
Zhang, Y., Jiang, Y. L., Qi, H., Zhao, M., Wang, Y., Wang, K., & Wei, F. (2022) · Sensors · doi:10.3390/s23010197 · Cited by 19 · see citing works
Recognition of Real-Time Hand Gestures using Mediapipe Holistic Model and LSTM with MLP Architecture
Amit, M. L., Fajardo, A. C., & Medina, R. P. (2022) · IEEE Conference on Systems, Process and Control · doi:10.1109/icspc55597.2022.10001800 · Cited by 22 · see citing works
One-shot learning hand gesture recognition based on modified 3d convolutional neural networks
Lu, Z., Qin, S., Li, X., Li, L., & Zhang, D. (2019) · Machine Vision and Applications · doi:10.1007/s00138-019-01043-7 · Cited by 19 · see citing works
Recognizing American Sign Language Manual Signs from RGB-D VideosPreprint
Jing, L., Vahdani, E., Huenerfauth, M., & Tian, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02851 · Cited by 19 · see citing works
3D Gesture Recognition and Adaptation for Human–Robot Interaction
Mahmud, J. A., Das, B. C., Shin, J., Hasib, K. M., Sadik, R., & Mridha, M. F. (2022) · IEEE Access · doi:10.1109/access.2022.3218679 · Cited by 17 · see citing works
Lu, Z., Qin, S., Li, L., Zhang, D., Xu, K., & Hu, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2940997 · Cited by 16 · see citing works
Benítez-García, G., Haris, M., Tsuda, Y., & Ukita, N. (2020) · IEEE transactions on intelligent transportation systems (Print) · doi:10.1109/tits.2020.3010306 · Cited by 13 · see citing works
Li, Y., Chen, H., Feng, G., & Miao, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01917 · Cited by 15 · see citing works
Self-citationProgression Modelling for Online and Early Gesture Detection
Gupta, V., Dwivedi, S. K., Dabral, R., & Jain, A. (2019) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2019.00040 · Cited by 15 · see citing works
Evidential dissonance measure in robust multi-view classification to resist adversarial attack
Yue, X., Dong, Z., Chen, Y., & Xie, S. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102605 · Cited by 14 · see citing works
High Performance Gesture Recognition via Effective and Efficient Temporal Modeling
Yi, Y., Ni, F., Ma, Y., Zhu, X., Qi, Y., Qiu, R., Zhao, S., Li, F., & Wang, Y. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/141 · Cited by 14 · see citing works
Adaptive cross-fusion learning for multi-modal gesture recognition
Zhou, B., Wan, J., Liang, Y., & Guo, G. (2021) · Virtual Reality & Intelligent Hardware · doi:10.1016/j.vrih.2021.05.003 · Cited by 13 · see citing works
Thermal video-based hand gestures recognition using lightweight CNN
Skriubakken, S. B., Fjeldvik, L. J., Noori, N. S., Yeduri, S. R., & Cenkeramaddi, L. R. (2024) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-024-04851-6 · Cited by 12 · see citing works
Fan, D., Yi, M., Kang, W., Wang, Y., & Lv, C. (2024) · Scientific Reports · doi:10.1038/s41598-024-78319-0 · Cited by 12 · see citing works
Hand gesture recognition based on attentive feature fusion
Yu, B., Luo, Z., Wu, H., & Li, S. (2020) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.5910 · Cited by 12 · see citing works
Li, L., Qin, S., Lu, Z., Zhang, D., Xu, K., & Hu, Z. (2021) · Pattern Analysis and Applications · doi:10.1007/s10044-021-00965-1 · Cited by 12 · see citing works
A deep co-attentive hand-based video question answering framework using multi-view skeleton
Rastgoo, R., Kiani, K., & Escalera, S. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13573-w · Cited by 12 · see citing works
Multimodal action recognition: a comprehensive survey on temporal modeling
Shabaninia, E., Nezamabadi–pour, H., & Shafizadegan, F. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17345-y · Cited by 28 · see citing works
Sign Language Recognition for Patient-Doctor Communication: A Multimedia/Multimodal Dataset
Mineo, R., Caligiore, G., Spampinato, C., Fontana, S., Palazzo, S., & Ragonese, E. (2024) · IEEE International Forum on Research and Technology for Society and Industry (RTSI) · doi:10.1109/rtsi61910.2024.10761720 · Cited by 11 · see citing works
Finger Gesture Spotting from Long Sequences Based on Multi-Stream Recurrent Neural Networks
Benítez-García, G., Haris, M., Tsuda, Y., & Ukita, N. (2020) · Sensors · doi:10.3390/s20020528 · Cited by 11 · see citing works
Wan, S., Yang, L., Ding, K., & Qiu, D. (2023) · IEEE Access · doi:10.1109/access.2023.3278100 · Cited by 11 · see citing works
Seeking a Hierarchical Prototype for Multimodal Gesture Recognition
Li, Y., Qi, T., Ma, Z., Quan, D., & Miao, Q. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3295811 · Cited by 11 · see citing works
Self-citationTalking With Your Hands: Scaling Hand Gestures and Recognition With CNNs
Köpüklü, O., Yao, R., & Rigoll, G. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00345 · Cited by 11 · see citing works
A Deep Curriculum Learning Semi-Supervised Framework for Remote Sensing Scene Classification
Zhang, Q., Chen, J., & Yuan, B. (2025) · Applied Sciences · doi:10.3390/app15010360 · Cited by 10 · see citing works
Gesture Recognition System for Isolated Word Sign Language Based on Key-Point Trajectory Matrix
Fakhfakh, S., & Jemâa, Y. B. (2018) · Computación y Sistemas · doi:10.13053/cys-22-4-3046 · Cited by 10 · see citing works
Deep signature-based isolated and large scale continuous gesture recognition approach
Mahmoud, R., Belgacem, S., & Omri, M. N. (2020) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2020.08.017 · Cited by 10 · see citing works
Multi-Modal Multi-Channel American Sign Language Recognition
Vahdani, E., Jing, L., Huenerfauth, M., & Tian, Y. (2023) · International Journal of Artificial Intelligence and Robotics Research · doi:10.1142/s2972335324500017 · Cited by 9 · see citing works
Towards wide-scale continuous gesture recognition model for in-depth and grayscale input videos
Mahmoud, R., Belgacem, S., & Omri, M. N. (2021) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-020-01227-y · Cited by 9 · see citing works
Learning a compact embedding for fine-grained few-shot static gesture recognition
Hu, Z., Qiu, F., Sun, H., Zhang, W., Ding, Y., Lv, T., & Fan, C. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-18430-6 · Cited by 8 · see citing works
TIM-SLR: a lightweight network for video isolated sign language recognition
Wang, F., Zhang, L., Hao, Y., & Han, S. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08873-7 · Cited by 8 · see citing works
Abnormal gesture recognition based on multi-model fusion strategy
Lin, C., Lin, X., Xie, Y., & Liang, Y. (2018) · Machine Vision and Applications · doi:10.1007/s00138-018-0969-0 · Cited by 8 · see citing works
A CNN Bi-LSTM based Multimodal Continuous Hand Gesture Recognition
Nayan, N., Ghosh, D., & Pradhan, P. M. (2022) · IEEE India Council International Subsections Conference (INDISCON) · doi:10.1109/indiscon54605.2022.9862878 · Cited by 8 · see citing works
Xie, H., Wang, J., Shao, B., Gu, J. N., & Li, M. (2019) · IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) · doi:10.1109/ismar-adjunct.2019.00-30 · Cited by 8 · see citing works
Multi-view and multi-scale behavior recognition algorithm based on attention mechanism
Zhang, D., Chen, C., Tan, F., Qian, B., Li, W., He, X., & Lei, S. (2023) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2023.1276208 · Cited by 7 · see citing works
Geng, L.-C., Chen, J., Tie, Y., Qi, L., & Liang, C. (2024) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2024.104364 · Cited by 7 · see citing works
Liao, L., Zhang, X., & Li, C. (2019) · IEEE International Conference on Visual Communications and Image Processing (VCIP) · doi:10.1109/vcip47243.2019.8965816 · Cited by 7 · see citing works
Robustness of Deep LSTM Networks in Freehand Gesture Recognition
Schak, M., & Gepperth, A. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-30508-6_27 · Cited by 7 · see citing works
Zhang, X., Li, S., Zeng, X., Lu, P., & Sun, W. (2025) · Computers · doi:10.3390/computers14100432 · Cited by 6 · see citing works
- Authors
- Miao, Q., Li, Y., Ouyang, W., Ma, Z., Xu, X., Shi, W., & Cao, X.
- Affiliations
- Xidian University (CN)
- Chinese University of Hong Kong (HK)
- Chinese Academy of Sciences (CN)
- Venue
- 2017 IEEE International Conference on Computer Vision Workshops (ICCVW)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 3047-3055
- ISSN
- 2473-9936 (ISSN-L)
- OpenAlex
- W2770472008
- Cited by
- 168
- References
- 50
- Citations by year
- 2026: 6 · 2025: 12 · 2024: 22 · 2023: 17 · 2022: 23 · 2021: 27 · 2020: 23 · 2019: 28 · 2018: 10
- FWCI
- 12.61
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Conference on Computer Vision Workshops (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
- Keywords
- gesture recognition, multimodal fusion, Retinex, median filter, canonical correlation analysis, feature fusion, illumination variation, noise reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
