The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Accurate, Robust, and Flexible Real-time Hand Tracking
Abstract
We present a new real-time hand tracking system based on a single depth camera. The system can accurately reconstruct complex hand poses across a variety of subjects. It also allows for robust tracking, rapidly recovering from any temporary failures. Most uniquely, our tracker is highly flexible, dramatically improving upon previous approaches which have focused on front-facing close-range scenarios. This flexibility opens up new possibilities for human-computer interaction with examples including tracking at distances from tens of centimeters through to several meters (for controlling the TV at a distance), supporting tracking using a moving depth camera (for mobile scenarios), and arbitrary camera placements (for VR headsets). These features are achieved through a new pipeline that combines a multi-layered discriminative reinitialization strategy for per-frame pose estimation, followed by a generative model-fitting stage. We provide extensive technical details and a detailed qualitative and quantitative analysis.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Real-time human pose recognition in parts from single depth images
Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., & Blake, A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995316 · Cited by 3,491
Self-citationA Hybrid of Genetic Algorithm and Particle Swarm Optimization for Recurrent Network Design
Juang, C.-F. (2004) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2003.818557 · Cited by 964 · cited 1×
“MODEL FITTING In order to optimize the golden energy and achieve an accurate hand pose estimate, we employ a model fitting algorithm that combines features of particle swarm optimization (PSO) and genetic algorithms (GA) in the spirit of [11, 17, 20].”
MethodEfficient model-based 3D tracking of hand articulations using Kinect
Oikonomidis, I., Kyriazis, N., & Argyros, A. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.101 · Cited by 874 · cited 6×
“MODEL FITTING In order to optimize the golden energy and achieve an accurate hand pose estimate, we employ a model fitting algorithm that combines features of particle swarm optimization (PSO) and genetic algorithms (GA) in the spirit of [11, 17, 20].”
BackgroundMethodInfluentialVision-based hand pose estimation: A review
Erol, A., Bebis, G., Nicolescu, M., Boyle, R. D., & Twombly, X. (2007) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.10.012 · Cited by 866 · cited 1×
“RGB input Early work relied on monocular RGB cameras, making the problem extremely challenging (see [8] for a survey).”
BackgroundVision as Bayesian inference: analysis by synthesis?
Yuille, A., & Kersten, D. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.05.002 · Cited by 849
Real-Time Continuous Pose Recovery of Human Hands Using Convolutional Networks
Tompson, J., Stein, M., Lecun, Y., & Perlin, K. (2014) · ACM Transactions on Graphics · doi:10.1145/2629500 · Cited by 801 · cited 3×
“[30] demonstrate impressive hand tracking results using deep neural networks to predict feature locations and IK to infer a skeleton.”
BackgroundMethodA Survey of Glove-Based Systems and Their Applications
Dipietro, L., Sabatini, A. M., & Dario, P. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2008.923862 · Cited by 691 · cited 1×
“RELATED WORK Given our aims described above, we avoid encumbering the user’s hand with data gloves [7], colored gloves [32], wearable cameras [13], or markers [38], all of which can be barriers for natural interaction.”
BackgroundReal-time hand-tracking with a color glove
Wang, R. Y., & Popović, J. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531369 · Cited by 609
Decision Forests for Computer Vision and Medical Image Analysis
(2013) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-1-4471-4929-3 · Cited by 595
Self-citationRealtime and Robust Hand Tracking from Depth
Qian, C., Sun, X., Wei, Y., Tang, X., & Sun, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.145 · Cited by 470 · cited 4×
“Particle ‘Aging’ A per-generation clustering step was recently proposed in [20] to reduce an effect called ‘particle collapse’ whereby all particles are pulled too close together to a bad solution.”
BackgroundMethodInfluentialSelf-citation
Research works printed but not linked: 5
- 10.5555/1949767.1949885 · doi:10.5555/1949767.1949885
- 10.5555/524467.796047 · doi:10.5555/524467.796047
- 10.5555/2532129.2532141 · doi:10.5555/2532129.2532141
- 10.5555/2354409.2354668 · doi:10.5555/2354409.2354668
- 10.5555/2421731.2421738 · doi:10.5555/2421731.2421738
Cited by
Playing for Data: Ground Truth from Computer Games
Richter, S. R., Vineet, V., Roth, S., & Koltun, V. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_7 · Cited by 1,815
“Rendered depth maps of parametric models have been used prominently in training leading systems for human pose estimation and hand tracking [42, 43].”
MethodHand Keypoint Detection in Single Images Using Multiview Bootstrapping
Simon, T., Joo, H., Matthews, I., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.494 · Cited by 1,119
“[16] demonstrated practical performance across a large range, but there are still difficult cases such as hand-hand interactions and handobject interactions.”
BackgroundObservation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking
Cao, J., Pang, J., Weng, X., Khirodkar, R., & Kitani, K. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00934 · Cited by 1,115
Romero (2017) · doi:10.1145/3130800.3130883 · Cited by 1,056
Soli: ubiquitous gesture sensing with millimeter wave radar
Lien, J., Gillian, N., Karagozler, M. E., Amihood, P., Schwesig, C., Olson, E., Raja, H., & Poupyrev, I. (2016) · ACM Transactions on Graphics · doi:10.1145/2897824.2925953 · Cited by 957
“Using depth information by employing either multiple cameras or a single depth camera can significantly improve precision [Sharp et al. 2015], but does not scale well to the small form factor required by wearable and mobile devices.”
Background
Metadata
- Authors
- Sharp, T., Keskin, C., Robertson, D., Taylor, J., Shotton, J., Kim, D., Rhemann, C., Leichter, I., Vinnikov, A., Wei, Y., Freedman, D., Kohli, P., Krupka, E., Fitzgibbon, A., & Izadi, S.
- Affiliations
- Microsoft Research (United Kingdom) (GB)
- Microsoft (United States) (US)
- Microsoft Research Asia (China) (CN)
- Venue
- Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2015-04-18
- Pages
- 3633-3642
- OpenAlex
- W2023633446
- Cited by
- 431
- References
- 35
- Citations by year
- 2026: 6 · 2025: 12 · 2024: 18 · 2023: 25 · 2022: 27 · 2021: 48 · 2020: 52 · 2019: 55 · 2018: 61 · 2017: 52 · 2016: 57 · 2015: 17
- FWCI
- 47.15
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Keywords
- hand tracking, real-time tracking, depth camera, hand pose estimation, reinitialization, human-computer interaction, mobile device tracking, virtual reality headsets, robust tracking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Accurate, Robust, and Flexible Real-time Hand Tracking. https://app.cochat.ai/papers/10.1145/2702123.2702179
- The paper
- Sharp, T., Keskin, C., Robertson, D., Taylor, J., Shotton, J., Kim, D., Rhemann, C., Leichter, I., Vinnikov, A., Wei, Y., Freedman, D., Kohli, P., Krupka, E., Fitzgibbon, A., & Izadi, S. (2015). Accurate, Robust, and Flexible Real-time Hand Tracking. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. https://doi.org/10.1145/2702123.2702179
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.