LightGuide: projected visualizations for hand movement guidance
The DOI matches its Crossref record; 33 of the 34 references it cites were found in a registry or an index.
Abstract
LightGuide is a system that explores a new approach to gesture guidance where we project guidance hints directly on a user's body. These projected hints guide the user in completing the desired motion with their body part which is particularly useful for performing movements that require accuracy and proper technique, such as during exercise or physical therapy. Our proof-of-concept implementation consists of a single low-cost depth camera and projector and we present four novel interaction techniques that are focused on guiding a user's hand in mid-air. Our visualizations are designed to incorporate both feedback and feedforward cues to help guide users through a range of movements. We quantify the performance of LightGuide in a user study comparing each of our on-body visualizations to hand animation videos on a computer display in both time and accuracy. Exceeding our expectations, participants performed movements with an average error of 21.6mm, nearly 85% more accurately than when guided by video.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The papers collectively suggest that Pick-to-Light visual guidance systems can be effective in various applications.”
From Poka Yoke in Smart Production Systems with Pick-to-Light Implementation to Increase Efficiency (2023) · cites this paper as [14]
Citations
Cites
Journal of personality and social psychology
Armstrong, J. S. (1988) · International Journal of Forecasting · doi:10.1016/0169-2070(88)90126-4 · Cited by 10,485
Iterative point matching for registration of free-form curves and surfaces
Zhang, Z. (1994) · International Journal of Computer Vision · doi:10.1007/bf01427149 · Cited by 2,402 · cited 1×
“To help analyze shape, we use the Iterative Closest Point (ICP) algorithm to register the user’s movements to our model paths [34].”
MethodModel-based object pose in 25 lines of code
Dementhon, D. F., & Davis, L. S. (1995) · International Journal of Computer Vision · doi:10.1007/bf01450852 · Cited by 1,009 · cited 1×
“Once we establish correspondences between the 2D points of the projector and the 3D points of the camera, we use the POSIT algorithm [7] to find the position and orientation of the projector.”
MethodKnowledge-based augmented reality
Feiner, S., Macintyre, B., & Seligmann, D. (1993) · Communications of the ACM · doi:10.1145/159544.159587 · Cited by 972 · cited 1×
“Task Guidance in Augmented Reality The field of Augmented Reality (AR) has shown a number of methods to provide guidance by using head-mounted displays or mobile devices to convey instructions that are superimposed on virtual or real world video feeds [1,8].”
BackgroundRaskar, R., Welch, G., Cutts, M., Lake, A., Stesin, L., & Fuchs, H. (1998) · Proceedings of the 25th annual conference on Computer graphics and interactive techniques - SIGGRAPH '98 · doi:10.1145/280814.280861 · Cited by 833 · cited 1×
“Augmenting Environments with Projectors Recent advancements in projection technology have made it possible to imbue user’s environments with projection capabilities [24,26,27,33].”
BackgroundSkinput: appropriating the body as an input surface
Harrison, C., Tan, D., & Morris, D. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753394 · Cited by 694
OmniTouch: wearable multitouch interaction everywhere
Harrison, C., Benko, H., & Wilson, A. D. (2011) · Proceedings of the 24th annual ACM symposium on User interface software and technology · doi:10.1145/2047196.2047255 · Cited by 562 · cited 4×
“To ensure that the visual hints do not move off of the user’s hand, we followed the same procedure as [12] in which we compute a derivative map of the depth image to check for large changes in the boundaries at the contours of the hand.”
BackgroundMethodResultInfluentialSelf-citationDistance Transforms of Sampled Functions
Felzenszwalb, P. F., & Huttenlocher, D. P. (2004) · eCommons (Cornell University) · Cited by 397
WUW - wear Ur world: a wearable gestural interface
Mistry, P., Maes, P., & Chang, L. (2009) · CHI '09 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1520340.1520626 · Cited by 372
Shader Lamps: Animating Real Objects With Image-Based Illumination
Raskar, R., Welch, G., Low, K., & Bandyopadhyay, D. (2001) · doi:10.2312/egwr/egwr01/089-101 · Cited by 327
Research works printed but not linked: 1
- 10.5555/647653.732300 · doi:10.5555/647653.732300
Cited by
Items 101–144 of 144
Investigating Augmented Reality for Adaptive Motor-Skill Training
Turakhia, D. G., Parent, M. G., Grossman, T., Glueck, M., & Lafreniere, B. (2025) · Graphics Interface · doi:10.1145/3769872.3769891 · Cited by 7
AR Guidance Design for Line Tracing Speed Control
Ceyssens, J., van Deurzen, B., Ruiz, G. A. R., Luyten, K., & Fiore, F. D. (2023) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar59233.2023.00122 · Cited by 5
“Peternel et al. [15] used projected visualizations on top of the hand to guide the user’s hands toward a specific target area.”
BackgroundUsing Audio Cues to Support Motion Gesture Interaction on Mobile Devices
Morrison-Smith, S., Hofmann, M. K., Li, Y., & Ruiz, J. R. (2016) · ACM Transactions on Applied Perception · doi:10.1145/2897516 · Cited by 4
Peres, B. J., Campos, P., & Azadegan, A. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-29381-9_12 · Cited by 4
A Testbed for Exploring Multi-Level Precueing in Augmented Reality
Liu, J.-S., Tversky, B., & Feiner, S. (2022) · IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) · doi:10.1109/vrw55335.2022.00121 · Cited by 4
Projecting Recorded Expert Hands at Real Size, at Real Speed, and onto Real Objects for Manual Work
Suzuki, G., Murase, T., & Fujii, Y. (2016) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/2876456.2879476 · Cited by 4
“To learn hand movements, LightGuide [8] projects gesture guidance hints directly onto a user’s hand and P.I.A.N.O [4] projects music notation onto the instrument.”
Exploring AR hand augmentations as error feedback mechanisms for enhancing gesture-based tutorials
Fidalgo, C. G., Yan, Y., Sousa, M., Jorge, J. A., & Lindlbauer, D. (2025) · Frontiers in Virtual Reality · doi:10.3389/frvir.2025.1574965 · Cited by 5
JollyGesture: Exploring Dual-Purpose Gestures and Gesture Guidance in VR Presentations
Park, G. W., Tang, A., & Chevalier, F. (2024) · Graphics Interface · doi:10.1145/3670947.3670965 · Cited by 6
Henderson, J., Jonker, T. R., Lank, E., Wigdor, D., & Lafreniere, B. (2022) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3517262 · Cited by 3
“For example, instructional videos, iconic representations, and interactive guidance are popular techniques that have been used to instruct a user how to perform spatial, 3D, or mid-air interactions [1, 8, 15, 20, 22, 38].”
BackgroundMethodComposites: A Tangible Interaction Paradigm for Visual Data Analysis in Design Practice
Subramonyam, H., Adar, E., & Drucker, S. M. (2022) · International Conference on Advanced Visual Interfaces (AVI) · doi:10.1145/3531073.3531091 · Cited by 3
Cueing Sequential 6DoF Rigid-Body Transformations in Augmented Reality
Liu, J.-S., Tversky, B., & Feiner, S. (2023) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar59233.2023.00050 · Cited by 3
Han, P.-H., Lin, J.-W., Hsieh, C.-H., Hsu, J.-H., & Hung, Y. (2018) · ACM SIGGRAPH Posters · doi:10.1145/3230744.3230776 · Cited by 3
“LightGuide [Sodhi et al. 2012] used a projector hanging from the ceiling and projected visual information on users’ hands to guide hand movements.”
MethodLearning Patient Transfers with Technology: A Qualitative Investigation of the Design Space
Dürr, M., Pfeil, U., Müller, J., Borowski, M., Gröschel, C., & Reiterer, H. (2019) · Mensch und Computer (MuC) · doi:10.1145/3340764.3340784 · Cited by 3
“LightGuide projects hints onto a user’s body to guide handmovements [36].”
BackgroundMorphGrip: Haptic Guidance Using a Shape-Changing Grip
Olaya-Figueroa, J. F., Caillet, A. C., Willms, J., Kurzweg, M., Darwiche, N., Dann, D., Streicher, F., & Wolf, K. (2024) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3701571.3701573 · Cited by 3
“Outside the therapy scope, LightGuide helps guide the hand during path-following by projecting visual cues directly onto the user’s hand, including arrow and path representations in 2D and 3D formats [28].”
Knibbe, J., Benko, H., & Wilson, A. D. (2015) · Cited by 2
Self-citationResch, S., Hanania, J.-G., Schwind, V., Völz, D., & Morillo, D. (2025) · IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) · doi:10.1109/vrw66409.2025.00139 · Cited by 2
Representation-Plurality In Multi-Touch Mobile Visual Programming For Music
Yang, Q., & Essl, G. (2015) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1181415 · Cited by 4
Digitally Augmenting the Physical Ground Space with Timed Visual Cues for Crutch-Assisted Walking
Peres, B. J., Campos, P., & Azadegan, A. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3312891 · Cited by 2
“Projecting the information directly in the body, helped the user to keep concentrated and not distracted by the external factors [6].”
BackgroundTempo Synchronization of Physical Activities with a Mixed-Reality Human-Machine-Interface
Chinchilla, S., Luces, J. V. S., & Hirata, Y. (2021) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio54168.2021.9739486 · Cited by 2
“They have been used for coaching the learning of simple motions such as hand gestures [12], [13] and limb motions [14], as well as full-body motions of physical activities such as Tai-Chi [15], ballet [16] and dance [17].”
MethodUser Interface Agents for Guiding Interaction with Augmented Virtual Mirrors
Lee, G., Rudhru, O., Park, H. S., Kim, H. W., & Billinghurst, M. (2017) · Eurographics · doi:10.2312/egve.20171347 · Cited by 2
“To overcome this limitation, there has been research on providing visual cues to help novice users learn gestures on desktop interfaces [BZW ∗ 09] [BM08], touch interaction [FBMW09] [DCN15], and motion recognition and training [SBW12] [WLF07] [AGMF13].”
BackgroundA Dataset of Naturally Occurring, Whole-Body Background Activity to Reduce Gesture ConflictsPreprint
Freeman, D., Jota, R., Vogel, D., Wigdor, D., & Balakrishnan, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1509.06109 · Cited by 2
“Using body gestures for explicit input has been extensively studied [1, 3, 13, 19, 29, 34, 36, 40].”
BackgroundProjection Mapping Technologies for Seamless Spatial Augmentation
Iwai, D. (2025) · Springer proceedings in physics · doi:10.1007/978-981-96-9782-3_30 · Cited by 1
APS: a 3D human body posture set as a baseline for posture guidance
Elsayed, H., Weigel, M., von Willich, J., Funk, M., & Mühlhäuser, M. (2019) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/3316782.3324012 · Cited by 1
Han, D., Heshmat, Y., & Neustaedter, C. (2020) · Proceedings · doi:10.20380/gi2020.04 · Cited by 1
“For example, video conferencing software may include on-screen visuals [79] to show patients where to move the camera to capture an area of one’s body, or augmented visuals [80], [81] to guide patients to perform certain actions.”
BackgroundHamanishi, N., & Rekimoto, J. (2020) · International Conference on Advanced Visual Interfaces (AVI) · doi:10.1145/3399715.3401952 · Cited by 1
Haptic feedback in freehand gesture interaction
Karvinen, J. (2015) · Tampere University Institutional Repository (Tampere University) · Cited by 1
Towards Next-Generation Remote Physiotherapy with Videoconferencing Tools
Tang, A., & Dillman, K. R. (2013) · doi:10.11575/prism/31067 · Cited by 1
“Guidance is the most promising feature that current commercial tools are providing [15], and lessons should be taken from these and other tools like MotionMA and LightGuide [21, 27].”
BackgroundDoing more without more : data fusion in human-computer interaction
Arslan, C. (2020) · theses.fr (ABES) · doi:10.70675/66c509fez7e62z4febz9d62zf454aab7beb4 · Cited by 1
Oh, S., Kwon, H., & So, H. (2014) · ACM Symposium on Spatial User Interaction (SUI) · doi:10.1145/2659766.2661228
“Recently, projecting visual information on various physical objects and human body has gained its popularity as a new interaction approach, and has been applied to various fields such as medical imaging, education, and computer interface [1, 2].”
BackgroundCompact and low-power computational 3D sensors for gestural input
Colaço, A. (2014) · DSpace@MIT (Massachusetts Institute of Technology)
Use of projector-camera system for human-computer interaction
Chung, R. H. Y., & Dai, J. (2012)
Body as a Workspace: Design for Remote Physiotherapy
Dillman, K. R., & Tang, A. (2015) · doi:10.11575/prism/31063
A Persuasive Approach in Using Visual Cues to Facilitate Mobility Using Forearm Crutches.
Peres, B. J., Campos, P., & Azadegan, A. (2019)
Evaluating an Early Motion Planning Feedforward Sensorimotor Mechanism for 2D Sketching
Kritopoulou, P., Manitsaris, S., & Manitsaris, A. (2019) · European Conference on Cognitive Ergonomics · doi:10.1145/3335082.3335089
Investigating Encoding and Perspective for Augmented Reality Motion Guidance
Kandel, J., Kasumarthi, S., Tsalikis, S., Duppen, C., Szafir, D. J., Lewek, M. D., Fuchs, H., & Szafir, D. A. (2025) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar67309.2025.00165
Dynamic Projection of Shitsukan
Watanabe, Y. (2026) · Springer eBooks · doi:10.1007/978-981-95-4762-3_12
Liu, X., Ye, D., Chen, P., Fu, Y., Li, X., & Li, K. (2026) · ACM Transactions on Applied Perception · doi:10.1145/3799698
“The Sleeve AR system extends the approach described in [9] by adding projected areas throughout the arm and the user’s surroundings that enhancement enables stroke patients to progressively replicate movements recorded by a therapist based on the visual feedback and auditory feedback provided.”
Daneshzand, F., Périn, C., & Carpendale, S. (2025) · Graphics Interface · doi:10.1145/3769872.3769875
“Feedforward implementations include motion path visualizations [4, 12, 30], posture indicators [1, 31], and gesture sequence guidance [13, 15].”
Métaphore d’interaction gestuelle en environnement virtuel : application à l’apprentissage de gestes
Jeanne, F. (2017) · theses.fr (ABES) · doi:10.70675/124e77b7z1c70z4ceaza6c1z65beca288f7b · Cited by 2
Embedded vs. Situated: An Evaluation of AR Facial Training Feedback
Nargund, A. A., Park, A., Höllerer, T., & Sra, M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791941 · Cited by 2
BeatriXR: Comprehensive and Adaptive Feedforward Support for Guidance in Virtual Reality EICS008
Artizzu, V., Luyten, K., Ruiz, G. A. R., & Spano, L. D. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3816419 · Cited by 1
Metadata
- Authors
- Sodhi, R., Benko, H., & Wilson, A.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Microsoft (United States) (US)
- Venue
- Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2012-05-05
- Pages
- 179-188
- OpenAlex
- W2120860586
- Semantic Scholar
- 1473939a714d2ff5e9ab925e69fd2e1037fd5d83
- Cited by
- 144
- References
- 34
- Citations by year
- 2026: 4 · 2025: 7 · 2024: 8 · 2023: 7 · 2022: 10 · 2021: 13 · 2020: 9 · 2019: 16 · 2018: 11 · 2017: 11 · 2016: 12 · 2015: 13 · 2014: 9 · 2013: 11 · 2012: 3
- FWCI
- 8.09
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- Microsoft Research
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Augmented Reality Applications
- Keywords
- physiotherapy, physical activity, movement accuracy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.