Pressing a Button You Cannot See: Evaluating Visual Designs to Assist Persons with Low Vision through Augmented Reality
Open accessThe DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index; 2 of the 14 sources its claims cite were checked for support.
Abstract
Partial vision loss occurs in several medical conditions and affects persons of all ages. It compromises many daily activities, such as reading, cutting vegetables, or identifying and accurately pressing buttons, e.g., on ticket machines or ATMs. Touchscreen interfaces pose a particular challenge because they lack haptic feedback from interface elements and often require people with impaired vision to rely on others for help. We propose a smartglasses-based solution to utilize the user’s residual vision. Together with visually-impaired individuals, we designed assistive augmentations for touchscreen interfaces and evaluated their suitability to guide attention towards interface elements and to increase the accuracy of manual inputs. We show that augmentations improve interaction performance and decrease cognitive load, particularly for unfamiliar interface layouts.
Overview
CoChat Summary
TL;DR
This paper evaluates five AR visual augmentations for helping people with low vision press touchscreen buttons more accurately and quickly. It finds that augmentations improve task completion and reduce perceived workload overall, but the best design varies across individuals and interface familiarity.
Key findings
- For UnfamiliarLayout, task completion time decreased significantly with augmentation: 5.7 s in baseline vs. 3.5 s with augmentation. For FamiliarLayout, time also decreased from 2.7 s to 1.9 s, but this was not significant.
- Error rates dropped with augmentation in both scenarios: FamiliarLayout 0.33 → 0.19 and UnfamiliarLayout 0.24 → 0.18; the paper says this reduction was significant for FamiliarLayout.
- NASA-TLX workload was lower with augmentation: 33% lower in FamiliarLayout and 41% lower in UnfamiliarLayout (46 → 30 and 55 → 36, respectively; p = 0.08 and p = 0.01). Mental demand and effort were also reduced; for UnfamiliarLayout, temporal and physical demand were additionally reduced.
- No augmentation type clearly outperformed the others at the group level: the paper reports no significant differences among augmentations, though preferences differed by participant; for example, 5/13 preferred Circle, 5/13 preferred SimpleCross, and 3/13 preferred Star.
Methods
- Design: Within-subject study with two scenarios: FamiliarLayout (a keyboard-like layout) and UnfamiliarLayout (same grid size/appearance but randomized letters). Each scenario had six conditions: baseline plus five augmentations.
- Sample/data: 16 participants (8 female, 8 male; mean age 54.9 years, SD 18.2), all with visual impairment; three participants could not use the technology as intended and were handled with a demonstration-like approach, and four participants could not complete baseline but completed augmented conditions.
- Analysis: Measured task completion time, error rate, and NASA-TLX workload; used planned contrasts, pairwise Welch t-tests, and Wilcoxon signed-rank tests for NASA-TLX.
Limitations
- The paper states that the HoloLens augmentable field of view is limited to 30° horizontally and 17.5° vertically, and that the device is heavy and quite large; it also notes that improved tracking close to a surface is needed.
- The authors note the approach may not suit people with large central visual field defects, those with difficulty perceiving screen-spanning augmentations as a whole, or those who need close-up inspection of the interface.
- Evident from the text: there is substantial interindividual variability, and some participants could not perceive the baseline interface at all, which means baseline comparisons used a reduced subset of participants for RQ1 and RQ3.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 Valid · 0 Not valid · 5 Not checked · 86 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For FamiliarLayout, this improvement is not significant (RQ1; Baseline M = 2.7s, SE = 1.9 vs. {Aug-mented} M = 1.9s,SE = 0.5).”
§ Pressing a Button You Cannot See: Evaluating Visual Designs to Assist Persons with Low Vision through Augmented Reality > 6.1 Task Completion Time and Error Rate
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“No participant chose ArrowheadCross or MovingArrowheadCross as their preferred augmentation; in fact, these were often reported as least favorite.”
§ Pressing a Button You Cannot See: Evaluating Visual Designs to Assist Persons with Low Vision through Augmented Reality > 6.2 Preferred Visualization and Workload
plainn = 13- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If the interface is unfamiliar to the user, the augmentation additionally reduces the perceived temporal (Z=2.3,p<0.01) and physical (Z=1.7,p<0.05) demand.”
§ Pressing a Button You Cannot See: Evaluating Visual Designs to Assist Persons with Low Vision through Augmented Reality > 6.2 Preferred Visualization and Workload
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Regarding RQ2, we did not find evidence that one type of augmentation provides better assistance than another when analysed across participants.”
§ Pressing a Button You Cannot See: Evaluating Visual Designs to Assist Persons with Low Vision through Augmented Reality > RQ3
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our results show that augmentations not only improve the accessibility of touchscreens in terms of speed and accuracy of inputs but also provide a unique solution for a subset of individuals who, without assistive AR, were not able to interact with the touchscreens at all.”
§ Pressing a Button You Cannot See: Evaluating Visual Designs to Assist Persons with Low Vision through Augmented Reality > 1 INTRODUCTION
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
39 more claims
Create a free account to see all 93 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 20
- It may state two separate claims as one. 34
- The quote only partly states this claim. 2
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Not confirmedz = 1.48reported p = .08 · recomputed p = .139
“(Z=1.48,p = .08”
- Not confirmedz = 2.31reported p = .01 · recomputed p = .021
“Z=2.31, p = .01”
- Not confirmedz = 2reported p = .03 · recomputed p = .046
“Z=2.0, p = .03”
- Not confirmedz = 2reported p = .03 · recomputed p = .046
“;Z=2.0, p = .03”
- Not confirmedz = 1.6reported p = .06 · recomputed p = .110
“Z=1.6, p = .06”
3 more recomputed tests
Create a free account to see every test the Stats check recomputed.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Citations
Cites
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
NASA-Task Load Index (NASA-TLX); 20 Years Later
Hart, S. G. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605000909 · Cited by 4,297 · cited 1×
“In addition to the cumulative score, the subscales of the NASATLX are commonly analyzed individually [9].”
BackgroundDiscovering statistics using R
(2012) · Choice Reviews Online · doi:10.5860/choice.50-2114 · Cited by 2,534
Nasa-task load index (NASA-TLX); 20 years later: (577632012-009)
Hart, S. G. (2006) · PsycEXTRA Dataset · doi:10.1037/e577632012-009 · Cited by 838
Kane, S. K., Jayant, C., Wobbrock, J. O., & Ladner, R. E. (2009) · Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/1639642.1639663 · Cited by 383 · cited 1×
“The expression of a desire for customizability is a common result in the evaluation of assistive technology [15].”
BackgroundThe detection of motion in the peripheral visual field
Mckee, S. P., & Nakayama, K. (1984) · Vision Research · doi:10.1016/0042-6989(84)90140-8 · Cited by 375 · cited 2×
“In the first phase, we developed a large set of augmentations based on related work in HCI [25, 35] and perceptual psychology [4, 21].”
BackgroundMethodSeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision
Zhao, Y., Cutrell, E., Holz, C., Morris, M. R., Ofek, E., & Wilson, A. D. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300341 · Cited by 207 · cited 1×
“, contrast enhancement and magnification [34], followed by a more versatile system for the assistance of individuals with VI in virtual reality (VR) in 2019 [32].”
BackgroundInvestigating touchscreen accessibility for people with visual impairments
McGookin, D., Brewster, S., & Jiang, W. (2008) · Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges · doi:10.1145/1463160.1463193 · Cited by 151 · cited 3×
“troublesome, as the users cannot determine the distance between finger and touchscreen or the location of the target element within it [20].”
BackgroundRenner, P., & Pfeiffer, T. (2017) · IEEE Symposium on 3D User Interfaces (3DUI) · doi:10.1109/3dui.2017.7893338 · Cited by 147 · cited 3×
“Augmentations can guide attention in strenuous tasks like object search for users with full visual acuity by displaying symbols as pointers [26], project the scene into a map [5], guide attention directly towards the object [25], or show arrows at the”
BackgroundMethodFacade: Auto-generating Tactile Interfaces to Appliances
Guo, A., Kim, J., Chen, X. '., Yeh, T., Hudson, S. E., Mankoff, J., & Bigham, J. P. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025845 · Cited by 122 · cited 1×
“Other options to increase the accessibility of touchscreens are the addition of tactile feedback by creating physical overlays [7, 14, 20] or active haptic feedback devices to accommodate interface changes [12, 31].”
BackgroundForeSee: A Customizable Head-Mounted Vision Enhancement System for People with Low Vision
Zhao, Y., Szpiro, S., & Azenkot, S. (2015) · Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility - ASSETS '15 · doi:10.1145/2700648.2809865 · Cited by 115
CueSee: exploring visual cues for people with low vision to facilitate a visual search task
Zhao, Y., Szpiro, S., Knighten, J., & Azenkot, S. (2016) · Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2971648.2971730 · Cited by 111
Touchplates: low-cost tactile overlays for visually impaired touch screen users
Kane, S. K., Morris, M. R., & Wobbrock, J. O. (2013) · Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/2513383.2513442 · Cited by 107 · cited 2×
“ical overlays [14, 20] add tactile information about the position of interface components.”
BackgroundAlbouys-Perrois, J., Laviole, J., Briant, C., & Brock, A. M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174203 · Cited by 107
MudPad: tactile feedback and haptic texture overlay for touch surfaces
Jansen, Y., Karrer, T., & Borchers, J. (2010) · ACM International Conference on Interactive Tabletops and Surfaces · doi:10.1145/1936652.1936655 · Cited by 102
Designing Haptic Assistive Technology for Individuals Who Are Blind or Visually Impaired
Pawluk, D. T. V., Adams, R. J., & Kitada, R. (2015) · IEEE Transactions on Haptics · doi:10.1109/toh.2015.2471300 · Cited by 101
VizLens: A Robust and Interactive Screen Reader for Interfaces in the Real World
Guo, A., Chen, X. '., Qi, H., White, S., Ghosh, S., Asakawa, C., & Bigham, J. P. (2016) · Proceedings of the 29th Annual Symposium on User Interface Software and Technology · doi:10.1145/2984511.2984518 · Cited by 99
Designing AR Visualizations to Facilitate Stair Navigation for People with Low Vision
Zhao, Y., Kupferstein, E., Castro, B. V., Feiner, S., & Azenkot, S. (2019) · Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3332165.3347906 · Cited by 97 · cited 2×
“Another solution helps users to navigate and use stairs more easily by highlighting the edges of the steps [33].”
BackgroundTowards Efficient Visual Guidance in Limited Field-of-View Head-Mounted Displays
Bork, F., Schnelzer, C., Eck, U., & Navab, N. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2868584 · Cited by 90
Gruenefeld, U., Ennenga, D., Ali, A. E., et al. (2017) · Symposium on Spatial User Interaction · doi:10.1145/3131277.3132175 · Cited by 89 · cited 1×
“Augmentations can guide attention in strenuous tasks like object search for users with full visual acuity by displaying symbols as pointers [26], project the scene into a map [5], guide attention directly towards the object [25], or show arrows at the”
BackgroundAn augmented reality sign-reading assistant for users with reduced vision
Huang, J., Kinateder, M., Dunn, M. J., Jarosz, W., Yang, X.-D., & Cooper, E. A. (2019) · PLOS ONE · doi:10.1371/journal.pone.0210630 · Cited by 75 · cited 2×
“propose an approach to recognize text on signs and display it in large and high-contrast virtual letters to the user [11].”
BackgroundScott, I. U., Feuer, W. J., & Jacko, J. A. (2002) · American Journal of Ophthalmology · doi:10.1016/s0002-9394(01)01406-4 · Cited by 62 · cited 1×
“show that persons with VI have difficulties selecting icons on a computer [30].”
BackgroundStearns, L., DeSouza, V., Yin, J., Findlater, L., & Froehlich, J. E. (2017) · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3132525.3134812 · Cited by 50 · cited 1×
“Sterns et al. proposed an electronic magnifier using the Microsoft HoloLens and a finger-mounted camera [27], which ports the functionality of an electronic magnifier to an HMD.”
MethodSiu, T., & Herskovic, V. (2013) · doi:10.1145/2535597.2535608 · Cited by 41 · cited 1×
“Augmentations can guide attention in strenuous tasks like object search for users with full visual acuity by displaying symbols as pointers [26], project the scene into a map [5], guide attention directly towards the object [25], or show arrows at the”
BackgroundStateLens: A Reverse Engineering Solution for Making Existing Dynamic Touchscreens Accessible
Guo, A., Kong, J., Rivera, M., Xu, F. F., & Bigham, J. P. (2019) · Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3332165.3347873 · Cited by 35 · cited 2×
“[8] developed a system using computer vision.”
MethodZhang, X., Tran, T., Sun, Y., Culhane, I., Jain, S., Fogarty, J., & Mankoff, J. (2018) · ASSETS '18 · doi:10.1145/3234695.3236349 · Cited by 30 · cited 2×
“Newer approaches use 3D printing to add tactile feedback to smartphones [31] or home appli-”
BackgroundScott, I. U., Feuer, W. J., & Jacko, J. A. (2002) · American Journal of Ophthalmology · doi:10.1016/s0002-9394(02)01795-6 · Cited by 27 · cited 1×
“decreasing the number of items displayed simultaneously [29].”
BackgroundUsing computer vision to access appliance displays
Fusco, G., Tekin, E., Ladner, R. E., & Coughlan, J. M. (2014) · Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility - ASSETS '14 · doi:10.1145/2661334.2661404 · Cited by 23 · cited 1×
“More recently, computer vision has been applied to the analysis of static and dynamically changing layouts [3, 8] and the guiding of manual inputs via auditory feedback [6].”
MethodLäng, F., Schmidt, A., & Machulla, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58796-3_19 · Cited by 23
Self-citationSupporting Older Adults in Using Complex User Interfaces with Augmented Reality
Kong, J., Guo, A., & Bigham, J. P. (2019) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3308561.3354593 · Cited by 12
Guo, A., Kim, J., Chen, X., YEH, T. P., Hudson, S. E., Mankoff, J. C., & Bigham, J. P. (2016) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2982142.2982187 · Cited by 4
Visualizing out-of-view objects in head-mounted augmented reality
Gruenefeld, U., Ali, A. E., Heuten, W., et al. (2017) · Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services · doi:10.1145/3098279.3122124
Jansen, Y., Karrer, T., & Borchers, J. (2010) · ACM International Conference on Interactive Tabletops and Surfaces · doi:10.1145/1936652.1936734
Research works printed but not linked: 6
- Anhong Guo Xiang Chen Haoran Qi Samuel White Suman Ghosh Chieko Asakawa and Jeffrey Bigham. 2016. VizLens: A Robust and Interactive Screen Reader for Interfaces in the Real World. 651-664. https://doi.org/10.1145/2984511.2984518 · DOI · Paper page
- Albouys Perrois Jérémy Jeremy Laviole Carine Briant and Anke Brock. 2018. Towards a Multisensory Augmented Reality Map for Blind and Low Vision People: a Participatory Design Approach. 1-14. https://doi.org/10.1145/3173574.3174203 · DOI · Paper page
- Patrick Renner and Thies Pfeiffer. 2017. Attention guiding techniques using peripheral vision and eye tracking for feedback in augmented-reality-based assistance systems. 186-194. https://doi.org/10.1109/3DUI.2017.7893338 · DOI
- Lee Stearns Victor DeSouza Jessica Yin Leah Findlater and Jon Froehlich. 2017. Augmented Reality Magnification for Low Vision Users with the Microsoft Hololens and a Finger-Worn Camera. 361-362. https://doi.org/10.1145/3132525.3134812 · DOI · Paper page
- Yuhang Zhao Sarit Szpiro and Shiri Azenkot. 2015. ForeSee: A Customizable Head-Mounted Vision Enhancement System for People with Low Vision. 239-249. https://doi.org/10.1145/2700648.2809865 · DOI · Paper page
- Yuhang Zhao Sarit Szpiro Jonathan Knighten and Shiri Azenkot. 2016. CueSee: exploring visual cues for people with low vision to facilitate a visual search task. 73-84. https://doi.org/10.1145/2971648.2971730 · DOI · Paper page
Cited by
Dudley, J., Yin, L., Garaj, V., & Kristensson, P. O. (2023) · Virtual Reality · doi:10.1007/s10055-023-00850-8 · Cited by 221
“Lang and Machulla (2021) evaluated a set of visual augmentations presented in AR, such as stationary and moving guidelines, designed to support visually impaired users with using touch screen interfaces.”
BackgroundInclusive AR/VR: accessibility barriers for immersive technologies
Creed, C., Al-Kalbani, M., Theil, A., Sarcar, S., & Williams, I. (2024) · Universal Access in the Information Society · doi:10.1007/s10209-023-00969-0 · Cited by 212
“Moreover, literature has explored the use of feed-forward techniques for immersive technology to support users with visual impairments in virtual interactions [55, 47, 56].”
BackgroundInclusive Augmented and Virtual Reality: A Research Agenda
Creed, C., Al-Kalbani, M., Theil, A., Sarcar, S., & Williams, I. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2247614 · Cited by 126
“The literature has also explored the use of feedforward techniques to support users with visual impairments in virtual interactions, notably in the use of visual enhancements to improve touch-based interactions (Coughlan & Miele, 2017; Lang & Machulla, 2021; Zhao et al., 2019).”
Jiang, C., Kuang, E., & Fan, M. (2025) · ACM Transactions on Accessible Computing · doi:10.1145/3711931 · Cited by 52
Lee, J., Tjahjadi, A. D., Kim, J., Yu, J., Park, M., Zhang, J., Froehlich, J. E., Tian, Y., & Zhao, Y. (2024) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3654777.3676449 · Cited by 46
Metadata
- Authors
- Lang, F., & Machulla, T.
- Affiliations
- LMU Munich, Germany
- TU Dortmund University (DE)
- ORCID
- Machulla, Tonja
- Venue
- Proceedings of the 27th ACM Symposium on Virtual Reality Software and Technology
- Publisher
- ACM
- Published
- 2021-12-08
- Pages
- 1-10
- OpenAlex
- W3215040104
- Cited by
- 32
- References
- 28
- Citations by year
- 2026: 6 · 2025: 13 · 2024: 5 · 2023: 5 · 2022: 3
- FWCI
- 1.01
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Symposium on Virtual Reality Software and Technology (VRST) (publishedVersion, cc-by-nc-sa)
- Funders
- Bundesministerium für Bildung und Forschung (16SV8102 IDeA)
- H2020 European Research Council (683008 AMPLIFY)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Augmented Reality Applications
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Keywords
- low vision, augmented reality, smart glasses, assistive technology, touch screen interface, haptic feedback, attention guidance, cognitive load, user interface layout, visual impairment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.