Analyzing visually impaired people’s touch gestures on smartphones
Open accessThe DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Abstract
We present an analysis of how visually impaired people perform gestures on touch-screen smartphones and report their preferences, explaining the procedure and technical implementation that we followed to collect gesture samples. To that end, we recruited 36 visually impaired participants and divided them into two main groups of low-vision and blind people respectively. We then examined their touch-based gesture preferences in terms of number of strokes, multi-touch, and shape angle, as well as their execution in geometric, kinematic and relative terms. For this purpose, we developed a wireless system to simultaneously record sample gestures from several participants, with the possibility of monitoring the capture process. Our results are consistent with previous research regarding the preference of visually impaired users for simple gestures: with one finger, a single stroke, and in one or two cardinal directions. Of the two groups of participants, blind people are less consistent with multi-stroke gestures. In addition, they are more likely than low-vision people to go outside the bounds of the display in the absence of its physical delimitation of, especially with multi-touch gestures. In the case of more complex gestures, rounded shapes are greatly preferred to angular ones, especially by blind people, who have difficulty performing straight gestures with steep or right angles. Based on these results and on previous related research, we offer suggestions to improve gesture accessibility of handheld touchscreen devices.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Online and off-line handwriting recognition: a comprehensive survey
Plamondon, R., & Srihari, S. N. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.824821 · Cited by 2,498
Kendall, M. G., & Smith, B. B. (1939) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177732186 · Cited by 944
Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes
Wobbrock, J. O., Wilson, A. D., & Li, Y. (2007) · Proceedings of the 20th annual ACM symposium on User interface software and technology · doi:10.1145/1294211.1294238 · Cited by 822
Specifying gestures by example
Rubine, D. (1991) · Proceedings of the 18th annual conference on Computer graphics and interactive techniques · doi:10.1145/122718.122753 · Cited by 778
User-defined motion gestures for mobile interaction
Ruiz, J., Li, Y., & Lank, E. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1978971 · Cited by 430
Mobile application development: web vs. native
Charland, A., & Leroux, B. (2011) · Communications of the ACM · doi:10.1145/1941487.1941504 · Cited by 423
SHARK 2: a large vocabulary shorthand writing system for pen-based computers
Kristensson, P.-O., & Zhai, S. (2004) · Proceedings of the 17th annual ACM symposium on User interface software and technology · doi:10.1145/1029632.1029640 · Cited by 419
Maximizing the guessability of symbolic input
Wobbrock, J. O., Aung, H. H., Rothrock, B., & Myers, B. A. (2005) · CHI '05 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1056808.1057043 · Cited by 416
A Comparative Study of Several Dynamic Time-Warping Algorithms for Connected-Word Recognition
Myers, C. S., & Rabiner, L. R. (1981) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1981.tb00272.x · Cited by 415
Oviatt, S., Cohen, P., Wu, L., Duncan, L., Suhm, B., Bers, J., Holzman, T., Winograd, T., Landay, J., Larson, J., et al. (2000) · Human–Computer Interaction · doi:10.1207/s15327051hci1504_1 · Cited by 375
Kane, S. K., Bigham, J. P., & Wobbrock, J. O. (2008) · Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/1414471.1414487 · Cited by 374 · cited 1×
“Nonetheless, touch-based interfaces may be a challenge for most visually impaired people, especially for users who have been totally blind since infancy [18].”
BackgroundGoldberg, D., & Richardson, C. (1993) · Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '93 · doi:10.1145/169059.169093 · Cited by 322
Usable gestures for blind people: understanding preference and performance
Kane, S. K., Wobbrock, J. O., & Ladner, R. E. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979001 · Cited by 314
High precision touch screen interaction
Albinsson, P.-A., & Zhai, S. (2003) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642631 · Cited by 279
Reducing legacy bias in gesture elicitation studies
Morris, M. R., Danielescu, A., Drucker, S., Fisher, D., Lee, B., schraefel, M. C., & Wobbrock, J. O. (2014) · Interactions · doi:10.1145/2591689 · Cited by 254
Gestures as point clouds: a $P recognizer for user interface prototypes
Vatavu, R.-D., Anthony, L., & Wobbrock, J. O. (2012) · Proceedings of the 14th ACM international conference on Multimodal interaction · doi:10.1145/2388676.2388732 · Cited by 247
A lightweight multistroke recognizer for user interface prototypes
Anthony, L., & Wobbrock, J. O. (2010) · Graphics Interface · Cited by 215
A review of technologies for sensing contact location on the surface of a display
Walker, G. (2012) · Journal of the Society for Information Display · doi:10.1002/jsid.100 · Cited by 191
Bayus, B. L., Jain, S., & Rao, A. G. (1997) · Journal of Marketing Research · doi:10.1177/002224379703400105 · Cited by 180
Protractor: a fast and accurate gesture recognizer
Li, Y. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753654 · Cited by 173
Experimental analysis of touch-screen gesture designs in mobile environments
Bragdon, A., Nelson, E., Li, Y., & Hinckley, K. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979000 · Cited by 172
Age-related differences in performance with touchscreens compared to traditional mouse input
Findlater, L., Froehlich, J. E., Fattal, K., Wobbrock, J. O., & Dastyar, T. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470703 · Cited by 172
Visual similarity of pen gestures
Long, A. C., Landay, J. A., Rowe, L. A., & Michiels, J. (2000) · Proceedings of the SIGCHI conference on Human Factors in Computing Systems · doi:10.1145/332040.332458 · Cited by 153
Exploring the use of speech input by blind people on mobile devices
Azenkot, S., & Lee, N. B. (2013) · Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/2513383.2513440 · Cited by 151 · cited 2×
“Besides, despite the great accessibility benefit of both platforms, they still have accessibility issues in terms of speech [6, 63] and gesture [26] recognition.”
BackgroundInvestigating touchscreen accessibility for people with visual impairments
McGookin, D., Brewster, S., & Jiang, W. (2008) · Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges · doi:10.1145/1463160.1463193 · Cited by 151 · cited 1×
“For instance, researchers are not always able to recruit visually impaired participants, particularly blind people, so they conduct their studies with blindfolded participants [28, 47] or by covering the device display [34].”
BackgroundHMM-based efficient sketch recognition
Sezgin, T. M., & Davis, R. (2005) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/1040830.1040899 · Cited by 147
Anthony, L., Brown, Q., Nias, J., Tate, B., & Mohan, S. (2012) · ACM International Conference on Interactive Tabletops and Surfaces (ITS) · doi:10.1145/2396636.2396671 · Cited by 131
Blind people and mobile touch-based text-entry: acknowledging the need for different flavors
Oliveira, J., Guerreiro, T., Nicolau, H., Jorge, J., & Gonçalves, D. (2011) · The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/2049536.2049569 · Cited by 123 · cited 1×
“Both sensory and cognitive capabilities have a significant impact on touch-based interaction performance [35].”
BackgroundTowards a formalization of multi-touch gestures
Kammer, D., Wojdziak, J., Keck, M., Groh, R., & Taranko, S. (2010) · ACM International Conference on Interactive Tabletops and Surfaces (ITS) · doi:10.1145/1936652.1936662 · Cited by 120
Postma, A., Zuidhoek, S., Noordzij, M. L., & Kappers, A. M. L. (2007) · Perception · doi:10.1068/p5441 · Cited by 110
The immediate usability of graffiti
MacKenzie, I. S., & Zhang, S. X. (1997) · Cited by 109
Touchplates: low-cost tactile overlays for visually impaired touch screen users
Kane, S. K., Morris, M. R., & Wobbrock, J. O. (2013) · Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/2513383.2513442 · Cited by 107
$N-protractor: a fast and accurate multistroke recognizer
Anthony, L., & Wobbrock, J. O. (2012) · Cited by 105
Getting Smartphones to Talkback: Understanding the Smartphone Adoption Process of Blind Users
Rodrigues, A., Montague, K., Nicolau, H., & Guerreiro, T. (2015) · Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility - ASSETS '15 · doi:10.1145/2700648.2809842 · Cited by 102
Representing users in accessibility research
Sears, A., & Hanson, V. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979268 · Cited by 96
Bayus, B. L., Jain, S., & Rao, A. G. (1997) · Journal of Marketing Research · doi:10.2307/3152064 · Cited by 96
Multimodal collaborative handwriting training for visually-impaired people
Plimmer, B., Crossan, A., Brewster, S. A., & Blagojevic, R. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357119 · Cited by 84
Iconic and multi-stroke gesture recognition
Willems, D. J. M., Niels, R., van Gerven, M. A. J., & Vuurpijl, L. G. (2009) · Pattern Recognition · doi:10.1016/j.patcog.2009.01.030 · Cited by 82
The challenges and potential of end-user gesture customization
Oh, U., & Findlater, L. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466145 · Cited by 71
JustSpeak: enabling universal voice control on Android
Zhong, Y., Raman, T. V., Burkhardt, C., Biadsy, F., & Bigham, J. P. (2014) · Proceedings of the 11th Web for All Conference · doi:10.1145/2596695.2596720 · Cited by 70
Oh, U., Kane, S. K., & Findlater, L. (2013) · ASSETS '13 · doi:10.1145/2513383.2513455 · Cited by 68
Moore, D. A. (1999) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1999.2828 · Cited by 59
Relative accuracy measures for stroke gestures
Vatavu, R.-D., Anthony, L., & Wobbrock, J. O. (2013) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2522848.2522875 · Cited by 59
Multi-Touch: A New Tactile 2-D Gesture Interface for Human-Computer Interaction
Westerman, W. C., Elias, J. G., & Hedge, A. (2001) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120104500612 · Cited by 54
Understanding the consistency of users' pen and finger stroke gesture articulation
Anthony, L., Vatavu, R.-D., & Wobbrock, J. O. (2013) · Cited by 50
Blind and Deaf Consumer Preferences for Android and iOS Smartphones
Morris, J., & Mueller, J. (2014) · Springer eBooks · doi:10.1007/978-3-319-05095-9_7 · Cited by 49
Making touch-based kiosks accessible to blind users through simple gestures
Sandnes, F. E., Tan, T. B., Johansen, A., Sulic, E., Vesterhus, E., & Iversen, E. R. (2011) · Universal Access in the Information Society · doi:10.1007/s10209-011-0258-4 · Cited by 47
Understanding Users' Perceived Difficulty of Multi-Touch Gesture Articulation
Rekik, Y., Vatavu, R.-D., & Grisoni, L. (2014) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2663204.2663273 · Cited by 44
Vatavu, R.-D., Anthony, L., & Wobbrock, J. O. (2014) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2663204.2663256 · Cited by 40
A Compositional Model for Gesture Definition
Spano, L. D., Cisternino, A., & Paternò, F. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-34347-6_3 · Cited by 33
Expanding touch input vocabulary by using consecutive distant taps
Heo, S., Gu, J., & Lee, G. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557234 · Cited by 32
Pittman, J. A. (1991) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/108844.108914 · Cited by 30
Designing a text entry multimodal keypad for blind users of touchscreen mobile phones
Buzzi, M. C., Buzzi, M., Leporini, B., & Trujillo, A. (2014) · Proceedings of the 16th international ACM SIGACCESS conference on Computers & accessibility - ASSETS '14 · doi:10.1145/2661334.2661354 · Cited by 28 · cited 1×
“Although successful, this approach might be less convenient given the current simple haptic technology of smartphones for more complicated tasks such as text entry and editing [10].”
BackgroundSelf-citationSchmidt, M., & Weber, G. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02710-9_64 · Cited by 20
Mobile text-entry models for people with disabilities
Guerreiro, T., Lagoá, P., Nicolau, H., Santana, P., & Jorge, J. (2008) · Proceedings of the 15th European conference on Cognitive ergonomics: the ergonomics of cool interaction · doi:10.1145/1473018.1473067 · Cited by 18 · cited 1×
“These include the small keys on portable keyboards, difficulty using text-to-speech feedback, or dictation in noisy environments; in addition, Braille keyboards are perceived as expensive or cumbersome to carry [6, 15].”
BackgroundExploring Visually Impaired People's Gesture Preferences for Smartphones
Buzzi, M. C., Buzzi, M., Leporini, B., & Trujillo, A. (2015) · Proceedings of the 11th Biannual Conference of the Italian SIGCHI Chapter · doi:10.1145/2808435.2808448 · Cited by 18
Self-citationGesture based interaction for visually-impaired people
Vidal, S., & Lefebvre, G. (2010) · Nordic Conference on Human-Computer Interaction · doi:10.1145/1868914.1869035 · Cited by 15
Understanding Touch and Motion Gestures for Blind People on Mobile Devices
Romano, M., Bellucci, A., & Aedo, I. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-22701-6_3 · Cited by 13
Heat maps as a usability tool for multi-touch interaction in mobile applications
Lettner, F., & Holzmann, C. (2012) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/2406367.2406427 · Cited by 13
Vatavu, R.-D., Casiez, G., & Grisoni, L. (2013) · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/2470654.2470692 · Cited by 12
Luthra, V., & Ghosh, S. K. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-20681-3_3 · Cited by 11
Improving Accessibility Design on Touchscreens
Xu, S. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-20681-3_15 · Cited by 3
Research works printed but not linked: 2
- Benyon D (1993) Accommodating individual differences through an adaptive user interface. Hum Factors Inform Technol 10:149
- Rivera J, van der Meulen R (2014) Gartner says sales of smartphones grew 20 percent in third quarter of 2014. Gartner, Egham
Cited by
Khan, A., & Khusro, S. (2021) · Universal Access in the Information Society · doi:10.1007/s10209-020-00733-8 · Cited by 177
“Blind people perform common activities either through direct exploration [18], text entry [19], gesture-based [20] and multi-touch interaction [21] on smartphones and smartwatches.”
BackgroundKhan, A., & Khusro, S. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-018-7094-y · Cited by 77
“Blind people perform common activities either through direct exploration [20], text-entry [47], gesture-based [9], and multiple selections [64] on a smartphone.”
BackgroundOpen Challenges of Blind People Using Smartphones
Rodrigues, A., Nicolau, H., Montague, K., Guerreiro, J., & Guerreiro, T. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1768672 · Cited by 53
“As such, researchers took a particular interest in first understanding gestural interaction for blind people (Buzzi et al., 2017; S. K. Kane et al., 2008, 2009; McGookin et al., 2008; Oh et al., 2013).”
BackgroundDesign Guidelines and Recommendations for Multimodal, Touchscreen-based Graphics
Gorlewicz, J. L., Tennison, J. L., Uesbeck, P. M., Richard, M. E., Palani, H. P., Stefik, A., Smith, D. W., & Giudice, N. A. (2020) · ACM Transactions on Accessible Computing · doi:10.1145/3403933 · Cited by 53
“We have observed (along with References [13, 15, 27]) that executing these types of gestures can be difficult for a VI user to perform due to the timing and spatial constraints of the gesture.”
BackgroundClarifying Agreement Calculations and Analysis for End-User Elicitation Studies
Vatavu, R.-D., & Wobbrock, J. O. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3476101 · Cited by 57
“Understanding how different participants (or, the same participant across trials) choose [6] or are able [16] to articulate gestures is key for informing the design of gesture recognizers that would prove tolerant to such differences in articulation compared to the canonical examples from their training sets.”
BackgroundMethod
Metadata
- Authors
- Buzzi, M. C., Buzzi, M., Leporini, B., & Trujillo, A.
- Affiliations
- Institute of Informatics and Telematics (IT)
- Istituto di Scienza e Tecnologie dell'Informazione "Alessandro Faedo" (IT)
- Corresponding author
- Buzzi, Maria Claudia
- Venue
- Multimedia Tools and Applications
- Published
- 2016-05-17
- Volume
- 76
- Issue
- 4
- Pages
- 5141-5169
- ISSN
- 1380-7501 (ISSN-L)
- 1573-7721
- OpenAlex
- W2400364057
- Cited by
- 53
- References
- 62
- Citations by year
- 2025: 1 · 2024: 6 · 2023: 6 · 2022: 7 · 2021: 13 · 2020: 8 · 2019: 4 · 2018: 3 · 2017: 5
- FWCI
- 2.55
- Type
- journal article
- Language
- en
- Open access
- Licence
- other-oa
- www.springer.com/tdm (tdm)
- Locations
- Multimedia Tools and Applications (publishedVersion)
- CINECA IRIS Institutial research information system (University of Pisa) (submittedVersion)
- ISTI Open Portal (submittedVersion, other-oa)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Keywords
- visually impaired users, touch gestures, smartphones, low vision, blindness, multi-touch gestures, stroke number, touchscreen devices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.