The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
EgoGesture: A New Dataset and Benchmark for Egocentric Hand Gesture Recognition
Abstract
Gesture is a natural interface in human-computer interaction, especially interacting with wearable devices, such as VR/AR helmet and glasses. However, in the gesture recognition community, it lacks of suitable datasets for developing egocentric (first-person view) gesture recognition methods, in particular in the deep learning era. In this paper, we introduce a new benchmark dataset named EgoGesture with sufficient size, variation, and reality to be able to train deep neural networks. This dataset contains more than 24 000 gesture samples and 3 000 000 frames for both color and depth modalities from 50 distinct subjects. We design 83 different static and dynamic gestures focused on interaction with wearable devices and collect them from six diverse indoor and outdoor scenes, respectively, with variation in background and illumination. We also consider the scenario when people perform gestures while they are walking. The performances of several representative approaches are systematically evaluated on two tasks: gesture classification in segmented data and gesture spotting and recognition in continuous data. Our empirical study also provides an in-depth analysis on input modality selection and domain adaptation between different scenes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
Generating Sequences With Recurrent Neural Networks
Graves, A. (2013) · arXiv · doi:10.48550/arxiv.1308.0850 · Cited by 2,894
Improving the Fisher Kernel for Large-Scale Image Classification
Perronnin, F., Sánchez, J., & Mensink, T. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_11 · Cited by 2,588
Mitra, S., & Acharya, T. (2007) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2007.893280 · Cited by 1,981
Visual interpretation of hand gestures for human-computer interaction: a review
Pavlovic, V. I., Sharma, R., & Huang, T. S. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.598226 · Cited by 1,791
Vision based hand gesture recognition for human computer interaction: a survey
Rautaray, S. S., & Agrawal, A. (2015) · Artificial Intelligence Review · doi:10.1007/s10462-012-9356-9 · Cited by 1,526
Large-scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G. D., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · Cited by 1,387
Real-time American sign language recognition using desk and wearable computer based video
Starner, T., Weaver, J., & Pentland, A. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.735811 · Cited by 1,335
Long-term Recurrent Convolutional Networks for Visual Recognition and Description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., & Darrell, T. (2014) · doi:10.21236/ada623249 · Cited by 968
Molchanov, P., Yang, X., Gupta, S., Kim, K., Tyree, S., & Kautz, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.456 · Cited by 695
Robust 3D Action Recognition with Random Occupancy Patterns
Wang, J., Liu, Z., Chorowski, J., Chen, Z., & Wu, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33709-3_62 · Cited by 471
Ohn-Bar, E., & Trivedi, M. M. (2014) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2014.2337331 · Cited by 455
Accurate, Robust, and Flexible Real-time Hand Tracking
Sharp, T., Keskin, C., Robertson, D., Taylor, J., Shotton, J., Kim, D., Rhemann, C., Leichter, I., Vinnikov, A., Wei, Y., et al. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702179 · Cited by 431
Lending A Hand: Detecting Hands and Recognizing Activities in Complex Egocentric Interactions
Bambach, S., Lee, S., Crandall, D. J., & Yu, C. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.226 · Cited by 408
Super Normal Vector for Activity Recognition Using Depth Sequences
Yang, X., & Tian, Y. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.108 · Cited by 388
Jiang, Y.-G., Wu, Z., Wang, J., Xue, X., & Chang, S.-F. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2670560 · Cited by 381
Learning discriminative representations from RGB-D video data
Liu, L., & Shao, L. (2013) · UEA Digital Repository (University of East Anglia) · Cited by 302
Canonical Correlation Analysis of Video Volume Tensors for Action Categorization and Detection
Kim, T.-K., & Cipolla, R. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.167 · Cited by 279
ChaLearn Looking at People RGB-D Isolated and Continuous Datasets for Gesture Recognition
Wan, J., Li, S. Z., Zhao, Y., Zhou, S., Guyon, I., & Escalera, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.100 · Cited by 276
A Real Time System For Dynamic Hand Gesture Recognition With A Depth Sensor
Kurakin, A., Liu, Z., & Zhang, Z. (2016) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.52481 · Cited by 254
ChaLearn Looking at People Challenge 2014: Dataset and Results
Escalera, S., Baró, X., Gonzàlez, J., Bautista, M. A., Madadi, M., Reyes, M., Ponce-López, V., Escalante, H. J., Shotton, J., & Guyon, I. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-16178-5_32 · Cited by 211
Inductive Robust Principal Component Analysis
Bao, B.-K., Liu, G., Xu, C., & Yan, S. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2192742 · Cited by 161
Understanding Everyday Hands in Action from RGB-D Images
Rogez, G., Supančič, J. S., & Ramanan, D. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.443 · Cited by 147
Evaluating spatiotemporal interest point features for depth-based action recognition
Zhu, Y. M., Chen, W., & Guo, G. (2014) · Image and Vision Computing · doi:10.1016/j.imavis.2014.04.005 · Cited by 121
Cao, C., Zhang, Y., Wu, Y., Lu, H., & Cheng, J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.406 · Cited by 109
Self-citationExplore Efficient Local Features from RGB-D Data for One-Shot Learning Gesture Recognition
Wan, J., Guo, G., & Li, S. Z. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2513479 · Cited by 104
Gesture Recognition in Ego-centric Videos Using Dense Trajectories and Hand Segmentation
Baraldi, L., Paci, F., Serra, G., Benini, L., & Cucchiara, R. (2014) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2014.107 · Cited by 103
Multi-layered Gesture Recognition with Kinect
Jiang, F., Zhang, S., Wu, S., Gao, Y., & Zhao, D. (2017) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-57021-1_13 · Cited by 96
Using Convolutional 3D Neural Networks for User-independent continuous gesture recognition
Camgöz, N. C., Hadfield, S., Koller, O., & Bowden, R. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7899606 · Cited by 92
Head-mounted gesture controlled interface for human-computer interaction
Memo, A., & Zanuttigh, P. (2016) · Multimedia Tools and Applications · doi:10.1007/s11042-016-4223-3 · Cited by 89
Hand Pose Estimation from Local Surface Normals
Wan, C., Yao, A., & Gool, L. V. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46487-9_34 · Cited by 87
Human Action Recognition in Unconstrained Videos by Explicit Motion Modeling
Jiang, Y., Dai, Q., Liu, W., Xue, X., & Ngo, C. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2456412 · Cited by 78
A Pointing Gesture Based Egocentric Interaction System: Dataset, Approach and Application
Huang, Y., Liu, X., Zhang, X., & Jin, L. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.53 · Cited by 72
Human Gesture Recognition on Product Manifolds
Lui, Y. M. (2017) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-57021-1_2 · Cited by 70
Karaman, S., Benois‐Pineau, J., Dovgalecs, V., Mégret, R., Pinquier, J., Andre-Obrecht, R., Gaëstel, Y., & Dartigues, J. (2012) · Multimedia Tools and Applications · doi:10.1007/s11042-012-1117-x · Cited by 61
ChAirGest: a challenge for multimodal mid-air gesture recognition for close HCI
Ruffieux, S., Lalanne, D., & Mugellini, E. (2013) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2522848.2532590 · Cited by 59
Multimodal Gesture Recognition Using Multi-stream Recurrent Neural Network
Nishida, N., & Nakayama, H. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-29451-3_54 · Cited by 53
Large-scale Continuous Gesture Recognition Using Convolutional Neural Networks
Wang, P., Li, W., Liu, S., et al. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7899600 · Cited by 53
ChaLearn multi-modal gesture recognition 2013
Escalera, S., Gonzàlez, J., Baró, X., Reyes, M., Guyon, I., Athitsos, V., Escalante, H. J., Sigal, L., Argyros, A., Sminchisescu, C., et al. (2013) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2522848.2532597 · Cited by 51
Karaman, S., Benois‐Pineau, J., Dovgalecs, V., Pinquier, J., Andre-Obrecht, R., Gaëstel, Y., & Dartigues, F. (2014) · Cited by 44
Semantic Event Fusion of Different Visual Modality Concepts for Activity Recognition
Crispim-Junior, C. F., Buso, V., Avgerinakis, K., Μεδίτσκος, Γ., Briassouli, A., Benois‐Pineau, J., Kompatsiaris, I. Y., & Brémond, F. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2537323 · Cited by 43
Metaphoric Hand Gestures for Orientation-Aware VR Object Manipulation With an Egocentric Viewpoint
Jang, Y., Jeon, I., Kim, T., & Woo, W. (2016) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2016.2611824 · Cited by 39
Language-Motivated Approaches to Action Recognition
Malgireddy, M. R., Nwogu, I., & Govindaraju, V. (2017) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-57021-1_5 · Cited by 36
Deep Dynamic Neural Networks for Gesture Segmentation and Recognition
Wu, D., & Shao, L. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-16178-5_39 · Cited by 33
González-Díaz, I., Buso, V., Benois‐Pineau, J., Bourmaud, G., Usseglio, G., Mégret, R., Gaëstel, Y., & Dartigues, J. (2015) · Springer eBooks · doi:10.1007/978-3-319-17963-6_9 · Cited by 7
Cited by
Two-Stream Adaptive Graph Convolutional Networks for Skeleton-Based Action Recognition
Shi, L., Zhang, Y., Cheng, J., & Lu, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01230 · Cited by 2,071
Self-citationSkeleton-Based Action Recognition With Directed Graph Neural Networks
Shi, L., Zhang, Y., Cheng, J., & Lu, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00810 · Cited by 930
Self-citationReal-time Hand Gesture Detection and Classification Using Convolutional Neural Networks
Kopuklu, O., Gunduz, A., Kose, N., & Rigoll, G. (2019) · 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019) · doi:10.1109/fg.2019.8756576 · Cited by 319
The Jester Dataset: A Large-Scale Video Dataset of Human Gestures
Materzynska, J., Berger, G., Bax, I., & Memisevic, R. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00349 · Cited by 260
MMTM: Multimodal Transfer Module for CNN Fusion
Vaezi Joze, H. R., Shaban, A., Iuzzolino, M. L., & Koishida, K. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01330 · Cited by 259
Metadata
- Authors
- Zhang, Y., Cao, C., Cheng, J., & Lu, H.
- Venue
- IEEE Transactions on Multimedia
- Published
- 2018-05
- Volume
- 20
- Issue
- 5
- Pages
- 1038-1050
- ISSN
- 1520-9210 (ISSN-L)
- 1941-0077
- OpenAlex
- W2791526950
- Cited by
- 372
- References
- 48
- FWCI
- 26.11
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.