The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Capturing Hands in Action Using Discriminative Salient Points and Physics Simulation
Open accessAbstract
Hand motion capture is a popular research field, recently gaining more attention due to the ubiquity of RGB-D sensors. However, even most recent approaches focus on the case of a single isolated hand. In this work, we focus on hands that interact with other hands or objects and present a framework that successfully captures motion in such interaction scenarios for both rigid and articulated objects. Our framework combines a generative model with discriminatively trained salient points to achieve a low tracking error and with collision detection and physics simulation to achieve physically plausible estimates even in case of occlusions and missing visual data. Since all components are unified in a single objective function which is almost everywhere differentiable, it can be optimized with standard optimization techniques. Our approach works for monocular RGB-D sequences as well as setups with multiple synchronized RGB cameras. For a qualitative and quantitative evaluation, we captured 29 sequences with a large variety of interactions and up to 150 degrees of freedom.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Computational Approach to Edge Detection
Canny, J. (1986) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1986.4767851 · Cited by 30,004
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · cited 1×
“To evaluate the performance of the detector, we follow the PASCAL-VOC protocol (Everingham et al, 2010).”
MethodA Mathematical Introduction to Robotic Manipulation
Murray, R. M., Li, Z., & Sastry, S. S. (2017) · CRC Press eBooks · doi:10.1201/9781315136370 · Cited by 6,694
Shape matching and object recognition using shape contexts
Belongie, S., Malik, J., & Puzicha, J. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.993558 · Cited by 6,306
Efficient variants of the ICP algorithm
Rusinkiewicz, S., & Levoy, M. (2002) · Proceedings Third International Conference on 3-D Digital Imaging and Modeling · doi:10.1109/im.2001.924423 · Cited by 3,945
Real-time human pose recognition in parts from single depth images
Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., & Blake, A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995316 · Cited by 3,491
Statistical Color Models with Application to Skin Detection
Jones, M. J., & Rehg, J. M. (2002) · International Journal of Computer Vision · doi:10.1023/a:1013200319198 · Cited by 1,542
Object modeling by registration of multiple range images
Chen, Y., & Medioni, G. (2002) · Proceedings. 1991 IEEE International Conference on Robotics and Automation · doi:10.1109/robot.1991.132043 · Cited by 1,409 · cited 2×
“…as a least squared error of point-to-point distances: Emodel→data(θ,D) = ∑ i ‖vi(θ)−Xi‖2 (6) An alternative to the point-to-point distance is the pointto-plane distance, which is commonly used for 3D reconstruction (Chen and Medioni, 1991; Rusinkiewicz and Levoy, 2001; Rusinkiewicz et al, 2002).”
BackgroundMethodEfficient model-based 3D tracking of hand articulations using Kinect
Oikonomidis, I., Kyriazis, N., & Argyros, A. (2011) · Procedings of the British Machine Vision Conference 2011 · doi:10.5244/c.25.101 · Cited by 874
Vision-based hand pose estimation: A review
Erol, A., Bebis, G., Nicolescu, M., Boyle, R. D., & Twombly, X. (2007) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.10.012 · Cited by 866
Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation
Lewis, J. P., Cordner, M., & Fong, N. (2000) · Proceedings of the 27th annual conference on Computer graphics and interactive techniques - SIGGRAPH '00 · doi:10.1145/344779.344862 · Cited by 853
Real-Time Continuous Pose Recovery of Human Hands Using Convolutional Networks
Tompson, J., Stein, M., Lecun, Y., & Perlin, K. (2014) · ACM Transactions on Graphics · doi:10.1145/2629500 · Cited by 801
A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach
Paris, S., & Durand, F. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-007-0110-8 · Cited by 797
Real-time hand-tracking with a color glove
Wang, R. Y., & Popović, J. (2009) · ACM Transactions on Graphics · doi:10.1145/1531326.1531369 · Cited by 609
Hough Forests for Object Detection, Tracking, and Action Recognition
Gall, J., Yao, A., Razavi, N., Van Gool, L., & Lempitsky, V. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.70 · Cited by 583
Self-citationReal-time 3D model acquisition
Rusinkiewicz, S., Hall-Holt, O., & Levoy, M. (2002) · Proceedings of the 29th annual conference on Computer graphics and interactive techniques · doi:10.1145/566570.566600 · Cited by 576
Collision Detection for Deformable Objects
Teschner, M., Kimmerle, S., Heidelberger, B., Zachmann, G., Raghupathi, L., Fuhrmann, A., Cani, M. ‐., Faure, F., Magnenat‐Thalmann, N., Strasser, W., et al. (2005) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2005.00829.x · Cited by 507
Partitioned Sampling, Articulated Objects, and Interface-Quality Hand Tracking
MacCormick, J., & Isard, M. (2000) · Lecture notes in computer science · doi:10.1007/3-540-45053-x_1 · Cited by 504
Automatic rigging and animation of 3D characters
Baran, I., & Popović, J. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276467 · Cited by 495 · cited 2×
“The skinning weights are computed using (Baran and Popović 2007).”
MethodRealtime and Robust Hand Tracking from Depth
Qian, C., Sun, X., Wei, Y., Tang, X., & Sun, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.145 · Cited by 470
Automatic rigging and animation of 3D characters
Baran, I., & Popović, J. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276467 · Cited by 455
Visual tracking of high DOF articulated structures: An application to human hand tracking
Rehg, J. M., & Kanade, T. (1994) · Lecture Notes in Computer Science · doi:10.1007/bfb0028333 · Cited by 453
Accurate, Robust, and Flexible Real-time Hand Tracking
Sharp, T., Keskin, C., Robertson, D., Taylor, J., Shotton, J., Kim, D., Rhemann, C., Leichter, I., Vinnikov, A., Wei, Y., et al. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702179 · Cited by 431
Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor
Kim, D., Hilliges, O., Izadi, S., Butler, A. D., Chen, J., Oikonomidis, I., & Olivier, P. (2012) · Proceedings of the 25th annual ACM symposium on User interface software and technology · doi:10.1145/2380116.2380139 · Cited by 428
Latent Regression Forest: Structured Estimation of 3D Articulated Hand Posture
Tang, D., Chang, H. J., Tejani, A., & Kim, T.-K. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.490 · Cited by 398
Model-based tracking of self-occluding articulated objects
Rehg, J. M., & Kanade, T. (2002) · Proceedings of IEEE International Conference on Computer Vision · doi:10.1109/iccv.1995.466882 · Cited by 392
Estimating 3D hand pose from a cluttered image
Athitsos, V., & Sclaroff, S. (2003) · 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. · doi:10.1109/cvpr.2003.1211500 · Cited by 379
Real-time 3D model acquisition
Rusinkiewicz, S., Hall-Holt, O., & Levoy, M. (2002) · ACM Transactions on Graphics · doi:10.1145/566654.566600 · Cited by 319
Shape context and chamfer matching in cluttered scenes
Thayananthan, A., Stenger, B., Torr, P. H. S., & Cipolla, R. (2003) · 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. · doi:10.1109/cvpr.2003.1211346 · Cited by 308
Hand Pose Estimation and Hand Shape Classification Using Multi-layered Randomized Decision Forests
Keskin, C., Kıraç, F., Kara, Y. E., & Akarun, L. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33783-3_61 · Cited by 284
Motion Capture of Hands in Action Using Discriminative Salient Points
Ballan, L., Taneja, A., Gall, J., Van Gool, L., & Pollefeys, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33783-3_46 · Cited by 280
Self-citationOikonomidis, I., Kyriazis, N., & Argyros, A. A. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126483 · Cited by 274
Model-Based 3D Hand Pose Estimation from Monocular Video
de La Gorce, M., Fleet, D. J., & Paragios, N. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.33 · Cited by 268
Model-based 3D tracking of an articulated hand
Stenger, B., Mendonca, P. R. S., & Cipolla, R. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990976 · Cited by 263
Interactive Markerless Articulated Hand Motion Tracking Using RGB and Depth Data
Sridhar, S., Oulasvirta, A., & Theobalt, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.305 · Cited by 247
Capturing natural hand articulation
Wu, Y., Lin, J. Y., & Huang, T. S. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937656 · Cited by 229
Fast and robust hand tracking using detection-guided optimization
Sridhar, S., Mueller, F., Oulasvirta, A., & Theobalt, C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298941 · Cited by 228
Tracking the articulated motion of two strongly interacting hands
Oikonomidis, I., Kyriazis, N., & Argyros, A. A. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247885 · Cited by 225
Towards 3D hand tracking using a deformable model
Heap, T., & Hogg, D. (2002) · Proceedings of the Second International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1996.557255 · Cited by 220
Real-Time Articulated Hand Pose Estimation Using Semi-supervised Transductive Regression Forests
Tang, D., Yu, T.-H., & Kim, T.-K. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.400 · Cited by 219
A Survey on Human Motion Analysis from Depth Data
Ye, M., Zhang, Q., Wang, L., Zhu, J., Yang, R., & Gall, J. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-44964-2_8 · Cited by 213
Self-citationTwist Based Acquisition and Tracking of Animal and Human Kinematics
Bregler, C., Malik, J., & Pullen, K. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000011203.00237.9b · Cited by 197
3D hand pose reconstruction using specialized mappings
Rosales, R., Athitsos, V., Sigal, L., & Sclaroff, S. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937543 · Cited by 190
Construction and animation of anatomically based human hand models
Albrecht, I., Haber, J., & Seidel, H. (2003) · doi:10.5555/846276.846290 · Cited by 185
(1991) · Elsevier eBooks · doi:10.1016/c2013-0-11551-6 · Cited by 167
Visual Hand Tracking Using Nonparametric Belief Propagation
Sudderth, E. B., Mandel, M. I., Freeman, W. T., & Willsky, A. S. (2005) · 2004 Conference on Computer Vision and Pattern Recognition Workshop · doi:10.1109/cvpr.2004.474 · Cited by 167
Holzer, S., Rusu, R. B., Dixon, M., Gedikli, S., & Navab, N. (2012) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2012.6385999 · Cited by 160
Combined Region and Motion-Based 3D Tracking of Rigid and Articulated Objects
Brox, T., Rosenhahn, B., Gall, J., & Cremers, D. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.32 · Cited by 152
Self-citationDistance Transforms of Sampled Functions
Felzenszwalb, P. F., & Huttenlocher, D. P. (2004) · eCommons (Cornell University) · Cited by 151
Tracking a hand manipulating an object
Hamer, H., Schindler, K., Koller-Meier, E., & Gool, L. V. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459282 · Cited by 149
DART: Dense Articulated Real-Time Tracking
Schmidt, T., Newcombe, R. A., & Fox, D. (2014) · Robotics: Science and Systems · doi:10.15607/rss.2014.x.030 · Cited by 144
Three-Dimensional Shape Knowledge for Joint Image Segmentation and Pose Tracking
Rosenhahn, B., Brox, T., & Weickert, J. (2006) · International Journal of Computer Vision · doi:10.1007/s11263-006-9965-3 · Cited by 131
3D Articulated Models and Multiview Tracking with Physical Forces
Delamarre, Q., & Faugeras, O. D. (2001) · Computer Vision and Image Understanding · doi:10.1006/cviu.2000.0892 · Cited by 129
Grasp Recognition for Programming by Demonstration
Ekvall, S., & Kragić, D. (2006) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2005.1570207 · Cited by 128
Hands in action: real-time 3D reconstruction of hands in interaction with objects
Romero, J., Kjellström, H., & Kragić, D. (2010) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/robot.2010.5509753 · Cited by 126
Monocular real-time 3D articulated hand pose estimation
Romero, J., Kjellström, H., & Kragić, D. (2009) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/ichr.2009.5379596 · Cited by 107
Scalable 3D Tracking of Multiple Interacting Objects
Kyriazis, N., & Argyros, A. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.438 · Cited by 98
User-Specific Hand Modeling from Monocular Depth Sequences
Taylor, J., Stebbing, R. V., Ramakrishna, V., Keskin, C., Shotton, J., Izadi, S., Hertzmann, A., & Fitzgibbon, A. W. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.88 · Cited by 92
Ballan, L., & Cortelazzo, G. M. (2008) · Cited by 85
Self-citationSmart particle filtering for high-dimensional tracking
Bray, M., Koller-Meier, E., & Gool, L. V. (2007) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2005.09.013 · Cited by 77
Regression-based Hand Pose Estimation from Multiple Cameras
de Campos, T. E., & Murray, D. W. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.252 · Cited by 72
An object-dependent hand pose prior from sparse training data
Hamer, H., Gall, J., Weise, T., & Gool, L. V. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540150 · Cited by 62
Self-citationPhysically Plausible 3D Scene Tracking: The Single Actor Hypothesis
Kyriazis, N., & Argyros, A. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.9 · Cited by 60
Real-Time Hand Tracking Using a Sum of Anisotropic Gaussians Model
Sridhar, S., Rhodin, H., Seidel, H., Oulasvirta, A., & Theobalt, C. (2014) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2014.37 · Cited by 58
Functional categorization of objects using real-time markerless motion capture
Gall, J., Fossati, A., & Gool, L. V. (2011) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2011.5995582 · Cited by 57
Self-citationEvolutionary Quasi-Random Search for Hand Articulations Tracking
Oikonomidis, I., Lourakis, M., & Argyros, A. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.437 · Cited by 55
Pons‐Moll, G., & Rosenhahn, B. (2011) · Springer eBooks · doi:10.1007/978-0-85729-997-0_9 · Cited by 50
Real-Time Hand Tracking Using a Sum of Anisotropic Gaussians Model
Sridhar, S., Rhodin, H., Seidel, H., Oulasvirta, A., & Theobalt, C. (2014) · MPG.PuRe (Max Planck Society) · Cited by 47
Collision Detection for Deformable Objects
Teschner, M., Kimmerle, S., Heidelberger, B., Zachmann, G., Raghupathi, L., Fuhrmann, A., Cani, M., Fauré, F., Magnenat‐Thalmann, N., Straßer, W., et al. (2004) · HAL · doi:10.2312/egst.20041028 · Cited by 43
Capturing Hand Motion with an RGB-D Sensor, Fusing a Generative Model with Salient Points
Tzionas, D., Srikantha, A., Aponte, P., & Gall, J. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-11752-2_22 · Cited by 31
Self-citationA Comparison of Directional Distances for Hand Pose Estimation
Tzionas, D., & Gall, J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40602-7_14 · Cited by 18
Self-citation
Cited by
Expressive Body Capture: 3D Hands, Face, and Body From a Single Image
Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A. A., Tzionas, D., & Black, M. J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01123 · Cited by 1,781
Self-citationHand Keypoint Detection in Single Images Using Multiview Bootstrapping
Simon, T., Joo, H., Matthews, I., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.494 · Cited by 1,119
Romero (2017) · doi:10.1145/3130800.3130883 · Cited by 1,056
GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB
Mueller, F., Bernard, F., Sotnychenko, O., Mehta, D., Sridhar, S., Casas, D., & Theobalt, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00013 · Cited by 521
First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations
Garcia-Hernando, G., Yuan, S., Baek, S., & Kim, T.-K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00050 · Cited by 519
Metadata
- Authors
- Tzionas, D., Ballan, L., Srikantha, A., Aponte, P., Pollefeys, M., & Gall, J.
- Venue
- International Journal of Computer Vision
- Published
- 2016-03-18
- Volume
- 118
- Issue
- 2
- Pages
- 172-193
- ISSN
- 0920-5691 (ISSN-L)
- 1573-1405
- OpenAlex
- W2227547437
- Cited by
- 277
- References
- 71
- FWCI
- 36.25
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.