AR Haptic-Audio Conversion for Non-Visual Product Understanding in Smartphone AR
The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Online marketplaces increasingly offer AR "view-in-room" features, yet these tools assume visual interaction, leaving blind and low vision (BLV) users unable to independently assess product shape, size, or material—key factors in purchase decisions. We first conducted a formative study with seven BLV participants to identify online shopping challenges and design requirements. Based on these findings, we built a smartphone AR prototype that converts 3D geometry into vibration patterns and announces material properties via speech synthesis, letting users “feel” virtual products by moving their phones. We evaluated our prototype with four BLV users across four feedback conditions: baseline text description only, haptics-only, audio-only, and combined haptic-audio. Preliminary results suggest that the combined condition may best support product understanding and purchase confidence, while haptic feedback showed promise for conveying shape. This work contributes a haptic-audio conversion approach that extends accessible mobile AR toward e-commerce product evaluation on commodity smartphones.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
VizWiz: nearly real-time answers to visual questions
Bigham, J. P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R. C., Miller, R., Tatarowicz, A., White, B., White, S., et al. (2010) · Proceedings of the 23nd annual ACM symposium on User interface software and technology · doi:10.1145/1866029.1866080 · Cited by 601
What Value Do Explicit High Level Concepts Have in Vision to Language Problems?
Wu, Q., Shen, C., Liu, L., Dick, A., & Van Den Hengel, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.29 · Cited by 452
Haptics: The Present and Future of Artificial Touch Sensation
Culbertson, H., Schorr, S. B., & Okamura, A. M. (2018) · Annual Review of Control, Robotics, and Autonomous Systems · doi:10.1146/annurev-control-060117-105043 · Cited by 431
Benko, H., Holz, C., Sinclair, M., & Ofek, E. (2016) · Proceedings of the 29th Annual Symposium on User Interface Software and Technology · doi:10.1145/2984511.2984526 · Cited by 298
Zhao, Y., Bennett, C. L., Benko, H., Cutrell, E., Holz, C., Morris, M. R., & Sinclair, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173690 · Cited by 245
Haptic-feedback support for cognitive mapping of unknown spaces by people who are blind
Lahav, O., & Mioduser, D. (2008) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2007.08.001 · Cited by 160
Crowdsourcing subjective fashion advice using VizWiz: challenges and opportunities
Burton, M. A., Brady, E. L., Brewer, R. N., Neylan, C., Bigham, J. P., & Hurst, A. K. (2012) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2384916.2384941 · Cited by 110
A guided tour in haptic audio visual environments and applications
Eid, M. A., Orozco, M., & Saddik, A. E. (2007) · International Journal of Advanced Media and Communication · doi:10.1504/ijamc.2007.013918 · Cited by 78
BrowseWithMe: An Online Clothes Shopping Assistant for People with Visual Impairments
Stangl, A. J., Kothari, E., Jain, S. D., Yeh, T., Grauman, K., & Gurari, D. (2018) · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3234695.3236337 · Cited by 66
Research works printed but not linked: 1
- Thomas Hermann, Andy Hunt, and John G. Neuhoff (Eds.). 2011. The Sonification Handbook. Logos Verlag Berlin.
Metadata
- Authors
- Tokida, S., Tsutsui, A., Kobori, N., Gillingham, M., & Ryskeldiev, B.
- Affiliations
- The University of Tokyo (JP)
- University of Tsukuba (JP)
- Mercari R4D, Tokyo, Japan
- Venue
- Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2026-04-13
- Pages
- 1-7
- OpenAlex
- W7153894522
- Cited by
- 0
- References
- 13
- FWCI
- 0.00
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Augmented Reality Applications
- Physical Sciences › Engineering › Mechanical Engineering › Teleoperation and Haptic Systems
- Keywords
- augmented reality, haptic feedback, audio feedback, non-visual perception, smartphone AR, product understanding, multisensory interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: AR Haptic-Audio Conversion for Non-Visual Product Understanding in Smartphone AR. https://app.cochat.ai/papers/10.1145/3772363.3798874
- The paper
- Tokida, S., Tsutsui, A., Kobori, N., Gillingham, M., & Ryskeldiev, B. (2026). AR Haptic-Audio Conversion for Non-Visual Product Understanding in Smartphone AR. Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3772363.3798874
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.