A glove-based system for studying hand-object manipulation via joint pose and force sensing
The DOI matches its Crossref record; 30 of the 31 references it cites were found in a registry or an index.
Abstract
We present a design of an easy-to-replicate glove-based system that can reliably perform simultaneous hand pose and force sensing in real time, for the purpose of collecting human hand data during fine manipulative actions. The design consists of a sensory glove that is capable of jointly collecting data of finger poses, hand poses, as well as forces on palm and each phalanx. Specifically, the sensory glove employs a network of 15 IMUs to measure the rotations between individual phalanxes. Hand pose is then reconstructed using forward kinematics. Contact forces on the palm and each phalanx are measured by 6 customized force sensors made from Velostat, a piezoresistive material whose force-voltage relation is investigated. We further develop an open-source software pipeline consisting of drivers and processing code and a system for visualizing hand actions that is compatible with the popular Raspberry Pi architecture. In our experiment, we conduct a series of evaluations that quantitatively characterize both individual sensors and the overall system, proving the effectiveness of the proposed design.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Vision based hand gesture recognition for human computer interaction: a survey
Rautaray, S. S., & Agrawal, A. (2015) · Artificial Intelligence Review · doi:10.1007/s10462-012-9356-9 · Cited by 1,526 · cited 1×
“While researchers can track hand pose based on perception [3], force estimation from vision using numerical differentiation methods [4], [5], or sophisticated physics-based soft-body simulation [6], [7], glove-based devices still have their own advantages, presenting convenient, integrated solutions that can be natura…”
BackgroundA Survey of Glove-Based Systems and Their Applications
Dipietro, L., Sabatini, A. M., & Dario, P. (2008) · IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2008.923862 · Cited by 691 · cited 1×
“Dipietro et al. provided a comprehensive survey of glove-based system designs and their application from 1970s to 2008 [8].”
BackgroundAffective Body Expression Perception and Recognition: A Survey
Kleinsmith, A., & Bianchi-Berthouze, N. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2012.16 · Cited by 607 · cited 1×
“Just as whole body sensing [2] is critical for the study of human movement, hand pose and force information is crucial to the investigation of manipulative tasks.”
BackgroundModeling the constraints of human hand motion
Lin, J., Wu, Y., & Huang, T. S. (2002) · Proceedings Workshop on Human Motion · doi:10.1109/humo.2000.897381 · Cited by 270 · cited 1×
“Joint limits: A commonly used closed form representation of the finger joints motion constraints [26] is adapted.”
BackgroundWolverine: A wearable haptic interface for grasping in virtual reality
Choi, I., Hawkes, E. W., Christensen, D. L., Ploch, C. J., & Follmer, S. (2016) · 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759169 · Cited by 216 · cited 1×
“Wolverine [16], adapts a DC motor and time-of-flight sensor into an exoskeleton structure in order to obtain hand pose without using a glove directly.”
BackgroundSoft curvature sensors for joint angle proprioception
Kramer, R. K., Majidi, C., Sahai, R., & Wood, R. J. (2011) · 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2011.6094701 · Cited by 174 · cited 1×
“Using curvature/flex sensors to measure finger flexion is an approach that has been proven to be effective [14], [15].”
BackgroundAssessment of hand kinematics using inertial and magnetic sensors
Kortier, H. G., Sluiter, V. I., Roetenberg, D., & Veltink, P. H. (2014) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/1743-0003-11-70 · Cited by 173 · cited 4×
“In this way, a more comprehensive representation of the hand pose is captured [10].”
BackgroundMethodInfluentialUnderstanding tools: Task-oriented object modeling, learning and recognition
Zhu, Y., Zhao, Y., & Zhu, S.-C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298903 · Cited by 144 · cited 1×
“While researchers can track hand pose based on perception [3], force estimation from vision using numerical differentiation methods [4], [5], or sophisticated physics-based soft-body simulation [6], [7], glove-based devices still have their own advantages, presenting convenient, integrated solutions that can be natura…”
BackgroundSelf-citationToward a modular soft sensor-embedded glove for human hand motion and tactile pressure measurement
Hammond, F. L., Menguc, Y., & Wood, R. J. (2014) · 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems · doi:10.1109/iros.2014.6943125 · Cited by 103 · cited 1×
“[17] designs a liquid-metal embedded elastomer sensor that can measure”
BackgroundRobust realtime physics-based motion control for human grasping
Zhao, W., Zhang, J., Min, J., & Chai, J. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508412 · Cited by 100 · cited 1×
“While researchers can track hand pose based on perception [3], force estimation from vision using numerical differentiation methods [4], [5], or sophisticated physics-based soft-body simulation [6], [7], glove-based devices still have their own advantages, presenting convenient, integrated solutions that can be natura…”
BackgroundLigorio, G., & Sabatini, A. (2013) · Sensors · doi:10.3390/s130201919 · Cited by 95 · cited 2×
“pose sensing accuracy using filtering [12] and estimation techniques like the extended Kalman filter [10], [11], [13].”
MethodVideo-based hand manipulation capture through composite motion control
Wang, Y., Min, J., Zhang, J., Liu, Y., Xu, F., Dai, Q., & Chai, J. (2013) · ACM Transactions on Graphics · doi:10.1145/2461912.2462000 · Cited by 83 · cited 1×
“While researchers can track hand pose based on perception [3], force estimation from vision using numerical differentiation methods [4], [5], or sophisticated physics-based soft-body simulation [6], [7], glove-based devices still have their own advantages, presenting convenient, integrated solutions that can be natura…”
BackgroundTowards force sensing from vision: Observing hand-object interactions to infer manipulation forces
Pham, T.-H., Kheddar, A., Qammaz, A., & Argyros, A. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298898 · Cited by 82 · cited 1×
“While researchers can track hand pose based on perception [3], force estimation from vision using numerical differentiation methods [4], [5], or sophisticated physics-based soft-body simulation [6], [7], glove-based devices still have their own advantages, presenting convenient, integrated solutions that can be natura…”
BackgroundGlove-Based Approach to Online Signature Verification
Kamel, N., Sayeed, M. S., & Ellis, G. A. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.32 · Cited by 82 · cited 1×
“Using curvature/flex sensors to measure finger flexion is an approach that has been proven to be effective [14], [15].”
BackgroundEdmonds, M., Gao, F., Xie, X., Liu, H., Qi, S., Zhu, Y., Rothrock, B., & Zhu, S. C. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206196 · Cited by 78 · cited 1×
“human demonstrations [27].”
Self-citationThimbleSense: A Fingertip-Wearable Tactile Sensor for Grasp Analysis
Battaglia, E., Bianchi, M., Altobelli, A., Grioli, G., Catalano, M. G., Serio, A., Santello, M., & Bicchi, A. (2015) · IEEE Transactions on Haptics · doi:10.1109/toh.2015.2482478 · Cited by 60 · cited 1×
“on the fingertip, a specific tactile sensor is available [19].”
BackgroundLee, B. W., & Shin, H. (2015) · IEEE Sensors Journal · doi:10.1109/jsen.2015.2480600 · Cited by 58 · cited 2×
“ment is conducted using a similar setup to that mentioned in [25].”
BackgroundMethodLow-cost, fast and accurate reconstruction of robotic and human postures via IMU measurements
Santaera, G., Luberto, E., Serio, A., Gabiccini, M., & Bicchi, A. (2015) · 2015 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2015.7139569 · Cited by 56 · cited 1×
“pose sensing accuracy using filtering [12] and estimation techniques like the extended Kalman filter [10], [11], [13].”
MethodHand Pose Estimation by Fusion of Inertial and Magnetic Sensing Aided by a Permanent Magnet
Kortier, H. G., Antonsson, J., Schepers, H. M., Gustafsson, F., & Veltink, P. H. (2015) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2014.2357579 · Cited by 52 · cited 1×
“pose sensing accuracy using filtering [12] and estimation techniques like the extended Kalman filter [10], [11], [13].”
MethodShu, T., Gao, X., Ryoo, M. S., & Zhu, S. C. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989197 · Cited by 43 · cited 1×
“Recent study also shows the haptic feedback is crucial for recognizing interactions [28], indicating potential applications in social interactions [29],”
BackgroundSelf-citationLearning Social Affordance for Human-Robot InteractionPreprint
Shu, T., Ryoo, M. S., & Zhu, S. C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.03692 · Cited by 43 · cited 1×
“Recent study also shows the haptic feedback is crucial for recognizing interactions [28], indicating potential applications in social interactions [29], [30].”
BackgroundSelf-citationFinger-gesture Recognition Glove using Velostat (ICCAS 2011)
Jeong, E., Lee, J., & Kim, D. (2011) · International Conference on Control, Automation and Systems (ICCAS) · Cited by 28
A pressure-redistributing insole using soft sensors and actuators
Low, J.-H., Khin, P. M., & Yeow, C. (2015) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2015.7139599 · Cited by 27 · cited 1×
“In recent years, Velostat, a piezoresistive conductive film, has become popular and is applied to pressure sensing in the fields of soft robotics [21], robotic touch perception [22], robotic tactile interaction [23], and gesture recognition [24].”
BackgroundTagne, G., Hénaff, P., & Grégori, N. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759125 · Cited by 26 · cited 1×
“Robots that imitate the behaviors of humans may enable more natural and friendly interactions with humans in manmade environments, as with robotic handshaking [1].”
BackgroundTouch-based admittance control of a robotic arm using neural learning of an artificial skin
Pugach, G., Melnyk, A., Tolochko, O. I., Pitti, A., & Gaussier, P. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759519 · Cited by 25 · cited 1×
“In recent years, Velostat, a piezoresistive conductive film, has become popular and is applied to pressure sensing in the fields of soft robotics [21], robotic touch perception [22], robotic tactile interaction [23], and gesture recognition [24].”
BackgroundFingertip force estimation via inertial and magnetic sensors in deformable object manipulation
Mohammadi, M., Baldi, T. L., Scheggi, S., & Prattichizzo, D. (2016) · IEEE Haptics Symposium · doi:10.1109/haptics.2016.7463191 · Cited by 18 · cited 1×
“[20].”
Smart Fur Tactile Sensor for a Socially Assistive Mobile Robot
Müller, S., Schröter, C., & Gross, H.-M. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-22876-1_5 · Cited by 11 · cited 1×
“In recent years, Velostat, a piezoresistive conductive film, has become popular and is applied to pressure sensing in the fields of soft robotics [21], robotic touch perception [22], robotic tactile interaction [23], and gesture recognition [24].”
BackgroundForward kinematics using IMU on-body sensor network for mobile analysis of human kinematics
Taylor, T., Ko, S., Mastrangelo, C. H., & Bamberg, S. J. M. (2013) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2013.6609729 · Cited by 9 · cited 1×
“[9] tests a Zigbee network of IMUs using independent state estimation for feasibility of joint prediction.”
MethodFine manipulative action recognition through sensor fusion
Gu, Y., Sheng, W., Liu, M., & Ou, Y. (2015) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2015.7353476 · Cited by 7 · cited 1×
“[18], a glove equipped with FlexiForce sensors is used to collect force information, and a Vicon motion capture system is employed to track wrist, index finger, and thumb angles for manipulative action recognition.”
MethodCritical Features of Joint Actions that Signal Human Interaction.
Shu, T., Thurman, S. M., Chen, D., Zhu, S. C., & Lu, H. (2016) · eScholarship (California Digital Library) · Cited by 6
Self-citation
Cited by
DexPilot: Vision-Based Teleoperation of Dexterous Robotic Hand-Arm System
Handa, A., Van Wyk, K., Yang, W., Liang, J., Chao, Y.-W., Wan, Q., Birchfield, S., Ratliff, N., & Fox, D. (2020) · 2020 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra40945.2020.9197124 · Cited by 209
“For example, [18] and [19] showed that low-cost solutions for teleoperation with a glove can be constructed, though they tend to be quite bulky and have wires tethered to them.”
BackgroundA tale of two explanations: Enhancing human trust by explaining robot behavior
Edmonds, M., Gao, F., Liu, H., Xie, X., Qi, S., Rothrock, B. A., Zhu, Y., Wu, Y. N., Lu, H., & Zhu, S. C. (2019) · Science Robotics · doi:10.1126/scirobotics.aay4663 · Cited by 184
“We used a tactile glove with force sensors (28) to capture both the poses and the forces involved in human demonstrations in opening medicine bottles that require a visually latent interaction between the hand and the cap, e.”
MethodSelf-citationDark, Beyond Deep: A Paradigm Shift to Cognitive AI with Humanlike Common Sense
Zhu, Y., Gao, T., Fan, L., Huang, S., Edmonds, M., Liu, H., Gao, F., Zhang, C., Qi, S., Wu, Y. N., et al. (2020) · Engineering · doi:10.1016/j.eng.2020.01.011 · Cited by 132
“By using the Velostat [294] glove in demonstration, the fine forces used to unlock the child-safety mecha- nism become observable.”
BackgroundMethodInfluentialSelf-citationAnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System
Qin, Y., Yang, W., Huang, B., Wyk, K., Su, H., Wang, X., Chao, Y.-W., & Fox, D. (2023) · Robotics: Science and Systems XIX · doi:10.15607/rss.2023.xix.015 · Cited by 109
“Compared to the costly wearable hand tracking solutions, such as gloves [29, 30, 37], marker-based motion capture systems [68, 31], inertia sensors [66] or VR headsets [4, 28, 51, 41], vision-based hand tracking is particularly favorable due to its low cost and low intrusion to the human operator.”
BackgroundHigh-Fidelity Grasping in Virtual Reality using a Glove-based System
Liu, H., Zhang, Z., Xie, X., Zhu, Y., Liu, Y., Wang, Y., & Zhu, S.-C. (2019) · 2019 International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2019.8794230 · Cited by 73
“The glove designs utilizing Inertial sensors, mainly IMUs [16], [17], provide a more elegant solution as its setup is usually light enough that would afford natural movements.”
BackgroundMethodSelf-citation
Metadata
- Authors
- Liu, H., Xie, X., Millar, M., Edmonds, M., Gao, F., Zhu, Y., Santos, V. J., Rothrock, B., & Zhu, S.-C.
- Affiliations
- University of California, Los Angeles (US)
- Jet Propulsion Laboratory (US)
- Venue
- 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
- Publisher
- IEEE
- Published
- 2017-09
- Pages
- 6617-6624
- ISSN
- 2153-0858 (ISSN-L)
- OpenAlex
- W2775357942
- Cited by
- 60
- References
- 31
- Citations by year
- 2026: 2 · 2025: 11 · 2024: 10 · 2023: 6 · 2022: 7 · 2021: 4 · 2020: 6 · 2019: 8 · 2018: 4 · 2017: 2
- FWCI
- 4.13
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (publishedVersion)
- Funders
- Defense Advanced Research Projects Agency
- Multidisciplinary University Research Initiative
- Office of Naval Research
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
- Life Sciences › Neuroscience › Cognitive Neuroscience › Motor Control and Adaptation
- Keywords
- sensory glove, hand pose estimation, force sensing, inertial measurement unit, Velostat, piezoresistive force sensor, forward kinematics, open source software, Raspberry Pi
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.