Write-it-Yourself with the Aid of Smartwatches: A Wizard-of-Oz Experiment with Blind People
Open accessAbstract
Working with non-digital, standard printed materials has always been a challenge for blind people, especially writing. Blind people very often depend on others to fill out printed forms, write checks, sign receipts and documents. Extant assistive technologies for working with printed material have exclusively focused on reading, with little to no support for writing. Also, these technologies employ special-purpose hardware that are usually worn on fingers, making them unsuitable for writing. In this paper, we explore the idea of using off-the-shelf smartwatches (paired with smartphones) to assist blind people in both reading and writing paper forms including checks and receipts. Towards this, we performed a Wizard-of-Oz evaluation of different smartwatch-based interfaces that provide user-customized audio-haptic feedback in real-time, to guide blind users to different form fields, narrate the field labels, and help them write straight while filling out these fields. Finally, we report the findings of this study including the technical challenges and user expectations that can potentially inform the design of Write-it-Yourself aids based on smartwatches.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Shopping preparation makes it easy for users to buy from a list.
“It makes easy for users to buy from a list”
From Making Shopping Easy for People with Visual Impairment Using Mobile Assistive Technologies (2019) · cites this paper as [93]
The Introduction mentions online shopping and shopping receipts, but not preparing a shopping list or making purchases from one. The Method > Procedure describes writing on checks, receipts, and forms. Searches for shopping preparation and buying from a list found no passage addressing the claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Rise of Consumer Health Wearables: Promises and Barriers
Piwek, L., Ellis, D. A., Andrews, S., & Joinson, A. (2016) · PLoS Medicine · doi:10.1371/journal.pmed.1001953 · Cited by 1,301 · cited 1× · see citing works
“Our choice of smartwatches for the study was dictated by its growing popularity as the preferred wearable [23].”
Bigham, J. P., Jayant, C., Ji, H., Little, G., Miller, A. E., Miller, R. C., Tatarowicz, A., White, B., White, S., & YEH, T. P. (2010) · International Cross-Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1805986.1806020 · Cited by 536 · see citing works
Tactons: structured tactile messages for non-visual information display
Brewster, S. A., & Brown, L. M. (2004) · ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) · Cited by 534 · see citing works
Vibrotactile Display: Perception, Technology, and Applications
Choi, S., & Kuchenbecker, K. J. (2013) · Proceedings of the IEEE · doi:10.1109/jproc.2012.2221071 · Cited by 496 · cited 1× · see citing works
“These works use some combination of synthesized speech and sonification and haptic patterns for effective discrimination [10, 12, 14].”
ActiveBelt: Belt-Type Wearable Tactile Display for Directional Navigation
Tsukada, K., & Yasumura, M. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30119-6_23 · Cited by 317 · cited 1× · see citing works
“Audio-haptic cues for way-finding and for exploration of maps have been used in numerous projects [6, 11, 17, 18, 22, 24, 27].”
Wang, H., Lai, T. T.-T., & Choudhury, R. R. (2015) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/2789168.2790121 · Cited by 226 · cited 1× · see citing works
“For C2 and C3, ideas proposed in [7, 28] where information (e.g., typed word) is inferred from smartwatch sensor data, can be used.”
More than meets the eye: a survey of screen-reader browsing strategies
Borodin, Y., Bigham, J. P., Dausch, G., & Ramakrishnan, I. V. (2010) · Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1805986.1806005 · Cited by 222 · see citing works
Self-citationJayant, C., Ji, H., White, S., & Bigham, J. P. (2011) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2049536.2049573 · Cited by 168 · cited 1× · see citing works
“…poor lighting conditions; they require careful camera framing so that a target object is completely visible and centered within the camera’s field of view [15, 18]; they do not support complex documents and spatial data [16], and cannot determine which blocks of text to read, and in what order [9].”
Shilkrot, R., Huber, J., Wong, M. E., Maes, P., & Nanayakkara, S. C. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702421 · Cited by 102 · cited 4× · see citing works
“Besides, a bigger problem is that in some cases an AT solution, especially ones based on special-purpose hardware designed to be worn on fingers [25], interfere with writing.”
InfluentialRümelin, S., Rukzio, E., & Hardy, R. M. (2011) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/2047196.2047234 · Cited by 78 · cited 1× · see citing works
“Audio-haptic cues for way-finding and for exploration of maps have been used in numerous projects [6, 11, 17, 18, 22, 24, 27].”
A Tactile Compass for Eyes-Free Pedestrian Navigation
Pielot, M., Poppinga, B., Heuten, W., & Boll, S. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-23771-3_47 · Cited by 65 · cited 1× · see citing works
“Audio-haptic cues for way-finding and for exploration of maps have been used in numerous projects [6, 11, 17, 18, 22, 24, 27].”
Kane, S. K., Frey, B., & Wobbrock, J. O. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470704 · Cited by 63 · cited 1× · see citing works
“…poor lighting conditions; they require careful camera framing so that a target object is completely visible and centered within the camera’s field of view [15, 18]; they do not support complex documents and spatial data [16], and cannot determine which blocks of text to read, and in what order [9].”
Stearns, L., Du, R., Oh, U., Jou, C., Findlater, L., Ross, D. A., & Froehlich, J. E. (2016) · ACM Transactions on Accessible Computing · doi:10.1145/2914793 · Cited by 62 · cited 2× · see citing works
“A number of stand-alone reading applications, leveraging these advances, have emerged––examples include the pioneering Kurzweil Scanner [4] for desktops; KNFB Reader [3], Seeing AI [19] and Text Detective [5] for mobile phones; and FingerReader [25], OrCam [21], HandSight [26] for wearables devices.”
Manduchi, R., & Coughlan, J. M. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557328 · Cited by 43 · cited 2× · see citing works
“Audio-haptic cues for way-finding and for exploration of maps have been used in numerous projects [6, 11, 17, 18, 22, 24, 27].”
Evaluating Angular Accuracy of Wrist-based Haptic Directional Guidance for Hand Movement
Hong, J., Stearns, L., Froehlich, J. E., Ross, D., & Findlater, L. (2016) · Proceedings · doi:10.20380/gi2016.25 · Cited by 27 · cited 1× · see citing works
“These works use some combination of synthesized speech and sonification and haptic patterns for effective discrimination [10, 12, 14].”
Using wrist vibrations to guide hand movement and whole body navigation
Brock, A., Kammoun, S., Macé, M. J. ‐., & Jouffrais, C. (2014) · i-com · doi:10.1515/icom.2014.0026 · Cited by 23 · cited 1× · see citing works
“Audio-haptic cues for way-finding and for exploration of maps have been used in numerous projects [6, 11, 17, 18, 22, 24, 27].”
A Haptic Soundscape Map of the University of Oregon
Lawrence, M. M., Martinelli, N., & Nehmer, R. (2009) · Journal of Maps · doi:10.4113/jom.2009.1028 · Cited by 13 · cited 1× · see citing works
“Audio-haptic cues for way-finding and for exploration of maps have been used in numerous projects [6, 11, 17, 18, 22, 24, 27].”
Quick-glance and in-depth exploration of a tabletop map for visually impaired people
Bardot, S., Brock, A., Serrano, M., & Jouffrais, C. (2014) · Conference on l'Interaction Homme-Machine (IHM) · doi:10.1145/2670444.2670465 · Cited by 8 · cited 1× · see citing works
“Audio-haptic cues for way-finding and for exploration of maps have been used in numerous projects [6, 11, 17, 18, 22, 24, 27].”
Flagged first, then by citations
1 of the 19 references couldn’t be matched to a known paper.
Research works printed but not linked: 3
- Beltramelli T. and Risi S. Deep-Spying: Spying Using Smartwatch and Deep Learning. Retrieved from https://arxiv.org/abs/1512.05616. Beltramelli T. and Risi S. Deep-Spying: Spying Using Smartwatch and Deep Learning. Retrieved from https://arxiv.org/abs/1512.05616. · arXiv
- 10.5555/3076132.3076170 · DOI
- OrCam , 2017 . Orcam: See for Yourself. Retrieved from https://www.orcam.com/myeye/. OrCam, 2017. Orcam: See for Yourself. Retrieved from https://www.orcam.com/myeye/.
Cited by
Sorted by citations
Making Shopping Easy for People with Visual Impairment Using Mobile Assistive Technologies
Elgendy, M., Sik-Lanyi, C., & Kelemen, A. (2019) · Applied Sciences · doi:10.3390/app9061061 · Cited by 48 · see citing works
Gamage, B., Do, T.-T., Price, N. S. C., Lowery, A., & Marriott, K. (2023) · The 25th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3597638.3608955 · Cited by 39 · see citing works
“In fact, only one group of researchers have worked on the flling paper forms task, as reported in [26, 40, 41].”
Background"Hey Model!" – Natural User Interactions and Agency in Accessible Interactive 3D Models
Reinders, S., Butler, M., & Marriott, K. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376145 · Cited by 27 · see citing works
“[44], WoZ has been used to explore non-visual web access [4], smartwatch interfaces [9] and social assistive aids [36].”
BackgroundMethodTowards Enabling Blind People to Independently Write on Printed Forms
Feiz, S., Billah, S. M., Ashok, V. G., Shilkrot, R., & Ramakrishnan, I. V. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300530 · Cited by 21 · see citing works
“In fact, we explored this idea in a Wizard-of-Oz study [11], where we found that with little practice and customization, the participants could accurately interpret the tactile instructions.”
MethodSelf-citationBody Pose Sonification for a View-Independent Auditory Aid to Blind Rock Climbers
Ramsay, J. D., & Chang, H. J. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093462 · Cited by 12 · see citing works
“[7], uses an approach very similar to ours to enable blind people to fill out printed forms.”
MethodAccessWear: Making Smartphone Applications Accessible to Blind Users
Khanna, P., Feiz, S., Xu, J., Ramakrishnan, I. V., Jain, S., Bi, X., & Balasubramanian, A. (2023) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3570361.3592495 · Cited by 9 · see citing works
“Smartwatches are also used for interactions in navigation [18], writing [11], map exploration [9, 20], and Braille reading [14].”
Self-citationText entry for the Blind on Smartwatches: A study of Braille code input methods for a novel device
Luna, M. M., Nascimento, H. A. D., Quigley, A., & Soares, F. (2023) · Universal Access in the Information Society · doi:10.1007/s10209-022-00870-2 · Cited by 7 · see citing works
A Framework to Improve Web Form Accessibility for the Visually Impaired
Hakami, W. A. S., & Al-Aama, A. Y. (2023) · IEEE Access · doi:10.1109/access.2023.3330370 · Cited by 7 · see citing works
“In a different approach, researchers [15] utilized haptic feedback to help visually impaired users locate specific fields on a printed form.”
BackgroundMixed-method usability investigation of ARROWS: augmented reality for roadway work zone safety
Sabeti, S., Morris, N. L., & Shoghli, O. (2023) · International Journal of Occupational Safety and Ergonomics · doi:10.1080/10803548.2023.2295660 · Cited by 7 · see citing works
Interactive Form Filling Assistant on Smart Glasses for Blind Users
Khan, R. R., Sandu, S., & Hong, J. (2025) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663547.3759722 · Cited by 1 · see citing works
“Billah et al. [2] laid early groundwork with a Wizard-of-Oz study, showing that blind users could follow audio and haptic cues to write in designated areas.”
Eyes on the Palm: Investigating a Ring-Shaped Camera for Seamless Accessible Tactile Exploration
Tsutsui, A., Wang, X., Takagi, H., Ochiai, Y., & Asakawa, C. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791659 · Cited by 2 · see citing works
“WoZ methods are common in HCI to simulate not fully implemented functions, and prior work with BLV participants has noted that limited deception is sometimes necessary to preserve natu-ral behaviors [8, 12, 53].”
Is Querying Users Acceptable for Human Activity Recognition Based on Active Learning?
Presotto, R., Civitarese, G., & Bettini, C. (2023) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomworkshops56833.2023.10150407
“As another example, WOz has been adopted to design smart interfaces for blind people [18], [19].”
MethodAnwar, R., Macnaughton, J., & Latham, K. (2025) · Ophthalmic and Physiological Optics · doi:10.1007/s44402-026-00108-1
“Similar trade-offs have been reported in experimental studies: audio-haptic cues can improve spatial accuracy and legibility, but tend to increase mental effort and task duration [10, 27, 28].”
Making Videos Accessible for Blind and Low Vision Users Using a Multimodal Agent Video Player
Sinha, A. K., Olmos, A., Santos, R. D., Rodriguez, R. R., Landay, J. A., Sepah, S., Nelson, P., & Kane, S. K. (2026) · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion) · doi:10.1145/3776591.3833853
- Authors
- Billah, S. M., Ashok, V., & Ramakrishnan, IV
- Affiliations
- Stony Brook University (US)
- Venue
- 23rd International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2018-03-05
- Volume
- 2018
- Pages
- 427-431
- PMC
- PMC6049082
- OpenAlex
- W2792722892
- Cited by
- 14
- References
- 19
- Citations by year
- 2026: 2 · 2025: 2 · 2023: 5 · 2022: 1 · 2020: 2 · 2019: 2
- FWCI
- 1.72
- Type
- conference paper
- Language
- en
- Open access
- green
- europepmc.org
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Science Foundation
- National Institutes of Health
- National Eye Institute (R01EY026621)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Keywords
- smartwatches, assistive technology, blind users, writing support, paper forms, cheques, receipts, haptic guidance, Wizard of Oz evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
