Ally: Understanding Text Messaging to Build a Better Onscreen Keyboard for Blind People
The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Abstract
Millions of people worldwide use smartphones every day, but the standard issue QWERTY keyboard is poorly optimized for non-sighted input. In this article, we document the variety of methods blind people use to enter text into their smartphones, and focus on one particular need: sending text messages. We analyze two modern corpora of text messages and contrast them with an older text message corpus, as well as other corpora gathered from news articles, chat rooms, and books. We present a virtual keyboard for blind people optimized for sending text messages called Ally. To evaluate Ally, we conducted two user studies with blind participants. Our first study found increasing speeds and our second study found that half of participants reached comparable speeds to QWERTY, suggesting it may be a viable replacement. We conclude with a discussion of future work for non-sighted text-entry of text messages.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Other studies aimed to improve the usability of QWERTY keyboards for VIBPs (e.g., [38]) or to propose novel gestural (stroke-based) [39] or alternative layout keyboard-based input methods [40].”
From A Review of Design and Evaluation Practices in Mobile Text Entry for Visually Impaired and Blind Persons (2023) · cites this paper as [38]
In the citing paper
“Participant numbers ranged from as low as 2 [44] to up to 13 participants [38].”
From A Review of Design and Evaluation Practices in Mobile Text Entry for Visually Impaired and Blind Persons (2023) · cites this paper as [38]
In the citing paper
“We finally note that the majority of the prototypes involved some form of gesture support as the main method for entering characters (e.g., [36,38] or as a support mechanism, such as to change keyboard modes (from alphabetic to numeric) or to perform special actions such as backspacing or marking the end of a word.”
From A Review of Design and Evaluation Practices in Mobile Text Entry for Visually Impaired and Blind Persons (2023) · cites this paper as [38]
Citations
Cites
Phrase sets for evaluating text entry techniques
MacKenzie, I. S., & Soukoreff, R. W. (2003) · CHI '03 extended abstracts on Human factors in computing systems - CHI '03 · doi:10.1145/765891.765971 · Cited by 825
The design and evaluation of a high-performance soft keyboard
MacKenzie, I. S., & Zhang, S. X. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.302983 · Cited by 422
(2012) · Cited by 370
EdgeWrite: a stylus-based text entry method designed for high accuracy and stability of motion
Wobbrock, J. O., Myers, B. A., & Kembel, J. A. (2003) · Proceedings of the 16th annual ACM symposium on User interface software and technology · doi:10.1145/964696.964703 · Cited by 355
Quikwriting: continuous stylus-based text entry
Perlin, K. (1998) · Proceedings of the 11th annual ACM symposium on User interface software and technology · doi:10.1145/288392.288613 · Cited by 264
Measuring errors in text entry tasks: an application of the Levenshtein string distance statistic
Soukoreff, R. W., & MacKenzie, I. S. (2001) · CHI '01 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/634067.634256 · Cited by 241
Performance Optimization of Virtual Keyboards
Zhai, S., Hunter, M., & Smith, B. (2002) · Human-Computer Interaction · doi:10.1207/s15327051hci172&3_4 · Cited by 221
Theoretical upper and lower bounds on typing speed using a stylus and a soft keyboard
WILLIAM SOUKOREFF, R., & SCOTT MACKENZIE, I. (1995) · Behaviour & Information Technology · doi:10.1080/01449299508914656 · Cited by 211
Analysis of text entry performance metrics
Arif, A. S., & Stuerzlinger, W. (2009) · 2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH) · doi:10.1109/tic-sth.2009.5444533 · Cited by 206
Cirrin: a word-level unistroke keyboard for pen input
Mankoff, J., & Abowd, G. D. (1998) · Proceedings of the 11th annual ACM symposium on User interface software and technology · doi:10.1145/288392.288611 · Cited by 191
Privacy Concerns and Behaviors of People with Visual Impairments
Ahmed, T., Hoyle, R., Connelly, K., Crandall, D., & Kapadia, A. C. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702334 · Cited by 164
No-Look Notes: Accessible Eyes-Free Multi-touch Text Entry
Bonner, M. N., Brudvik, J. T., Abowd, G. D., & Edwards, W. K. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-12654-3_24 · Cited by 154 · cited 4×
“We were also originally inspired by several variations on multiple-stage text-entry [10] and optimizations for corpora [55] which helped inform our design and evaluation process.”
BackgroundMethodInfluentialExploring the use of speech input by blind people on mobile devices
Azenkot, S., & Lee, N. B. (2013) · Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/2513383.2513440 · Cited by 151 · cited 2×
“There are situations where blind users are reticent to use voice input for many of the same reasons as sighted users, including privacy, perceived accuracy, and the desire not to disturb others [1, 4].”
BackgroundBrailleTouch: Mobile Texting for the Visually Impaired
Frey, B., Southern, C., & Romero, M. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21666-4_3 · Cited by 129
Effects of Language Modeling and its Personalization on Touchscreen Typing Performance
Fowler, A., Partridge, K., Chelba, C., Bi, X., Ouyang, T., & Zhai, S. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702503 · Cited by 127 · cited 2×
“The presence of autocorrect means that misspellings and incorrect characters within words tend to be instantly automatically corrected using language models and personalized user history [17].”
BackgroundAn evaluation of BrailleTouch: mobile touchscreen text entry for the visually impaired
Southern, C., Clawson, J., Frey, B., Abowd, G., & Romero, M. (2012) · Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services · doi:10.1145/2371574.2371623 · Cited by 125
Blind people and mobile touch-based text-entry: acknowledging the need for different flavors
Oliveira, J., Guerreiro, T., Nicolau, H., Jorge, J., & Gonçalves, D. (2011) · The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/2049536.2049569 · Cited by 123 · cited 12×
“While recent research [37] suggests that speech recognition accuracy has improved, the laboratory conditions under which such research is conducted are significantly different from the noisy environs of a subway train or crowded office.”
BackgroundResultInfluentialPredictive text entry methods for mobile phones
Dunlop, M. D., & Crossan, A. (2000) · Personal and Ubiquitous Computing · doi:10.1007/bf01324120 · Cited by 121
BrailleType: Unleashing Braille over Touch Screen Mobile Phones
Oliveira, J., Guerreiro, T., Nicolau, H., Jorge, J., & Gonçalves, D. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-23774-4_10 · Cited by 111
Creating a live, public short message service corpus: The NUS SMS corpus
Chen, T., Kan, M., & Chen, T. (2011) · Cited by 105
Blindsight: eyes-free access to mobile phones
Li, K. A., Baudisch, P., & Hinckley, K. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357273 · Cited by 103
Getting Smartphones to Talkback: Understanding the Smartphone Adoption Process of Blind Users
Rodrigues, A., Montague, K., Nicolau, H., & Guerreiro, T. (2015) · Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility - ASSETS '15 · doi:10.1145/2700648.2809842 · Cited by 102
From Tapping to Touching: Making Touch Screens Accessible to Blind Users
Guerreiro, T., Lagoá, P., Nicolau, H., Gonalves, D., & Jorge, J. A. (2008) · IEEE Multimedia · doi:10.1109/mmul.2008.88 · Cited by 100
Yadav, K. S., Kumaraguru, P., Goyal, A., Gupta, A., & Naik, V. S. (2011) · IEEE Workshop on Mobile Computing Systems and Applications (WMCSA) · doi:10.1145/2184489.2184491 · Cited by 94
A large scale study of text-messaging use
Battestini, A., Setlur, V., & Sohn, T. (2010) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/1851600.1851638 · Cited by 90
Human Factors Engineering Studies of the Design and Use of Pushbutton Telephone Sets
Deininger, R. L. (1960) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1960.tb04447.x · Cited by 89
BlindType: Eyes-Free Text Entry on Handheld Touchpad by Leveraging Thumb's Muscle Memory
Lu, Y., Yu, C., Yi, X., Shi, Y., & Zhao, S. (2017) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3090083 · Cited by 76
Typing on an Invisible Keyboard
Zhu, S., Luo, T., Bi, X., & Zhai, S. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174013 · Cited by 75
Adaptive blind interaction technique for touchscreens
Yfantidis, G., & Evreinov, G. (2006) · Universal Access in the Information Society · doi:10.1007/s10209-004-0109-7 · Cited by 73 · cited 1×
“Others have explored alternative methods of inputting text, such as Blindsight, which provides auditory feedback for button presses during calls [24], an eyes-free shorthand handwriting recognition [49], and gestural input on adaptive sequences [53].”
MethodTypeInBraille: a braille-based typing application for touchscreen devices
Mascetti, S., Bernareggi, C., & Belotti, M. (2011) · The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/2049536.2049614 · Cited by 68
Recent developments in text-entry error rate measurement
Soukoreff, R. W., & MacKenzie, I. S. (2004) · CHI '04 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/985921.986081 · Cited by 65
Predicting the cost of error correction in character-based text entry technologies
Arif, A. S., & Stuerzlinger, W. (2010) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753329 · Cited by 55
Eyes-free text entry with error correction on touchscreen mobile devices
Tinwala, H., & MacKenzie, I. S. (2010) · Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries · doi:10.1145/1868914.1868972 · Cited by 51
Clawson, J., Lyons, K., Starner, T., & Clarkson, E. (2005) · Ninth IEEE International Symposium on Wearable Computers (ISWC'05) · doi:10.1109/iswc.2005.49 · Cited by 46
Language Use in Swedish Mobile Text Messaging
af Segerstad, Y. H. (2005) · Computer supported cooperative work · doi:10.1007/1-84628-248-9_21 · Cited by 44
NavTap: a long term study with excluded blind users
Guerreiro, T., Nicolau, H., Jorge, J., & Gonçalves, D. (2009) · Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/1639642.1639661 · Cited by 40
i'sFree: Eyes-Free Gesture Typing via a Touch-Enabled Remote Control
Zhu, S., Zheng, J., Zhai, S., & Bi, X. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300678 · Cited by 40
Typing Performance of Blind Users
Nicolau, H., Montague, K., Guerreiro, T., Rodrigues, A., & Hanson, V. L. (2015) · Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility - ASSETS '15 · doi:10.1145/2700648.2809861 · Cited by 37 · cited 1×
“studied text-entry errors in blind novice smartphone users and found that substitutions were the most common type of error and they had a substantive impact on text-entry [35].”
BackgroundSMS Language and College Writing :The languages of the College Texters
Shafie, L. A., Darus, N. A., & Osman, N. (2010) · International Journal of Emerging Technologies in Learning (iJET) · doi:10.3991/ijet.v5i1.1010 · Cited by 35
B#: chord-based correction for multitouch braille input
Nicolau, H., Montague, K., Guerreiro, T., Guerreiro, J., & Hanson, V. L. (2014) · International Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557269 · Cited by 34
Proficient blind users and mobile text-entry
Nicolau, H., Guerreiro, T., Jorge, J., & Gonçalves, D. (2010) · Proceedings of the 28th Annual European Conference on Cognitive Ergonomics · doi:10.1145/1962300.1962307 · Cited by 15 · cited 1×
“NavTap was found to be particularly helpful for novice blind users, who were found to perform fewer errors compared with using multi-tap entry [33].”
BackgroundFaster and Less Error-Prone: Supplementing an Accessible Keyboard with Speech Input
Bhikne, B., Joshi, A. N., Joshi, M., Jadhav, C., & Sakhardande, P. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-29381-9_18 · Cited by 11
How Much Faster Can You Type by Speaking in Hindi?
Bhikne, B., Joshi, A. N., Joshi, M., Ahire, S., & Maravi, N. (2018) · Indian Conference on Human-Computer Interaction (IndiaHCI) · doi:10.1145/3297121.3297123 · Cited by 9
Performance of Accessible Gesture-Based Indic Keyboard
Bharath, P. A., Jadhav, C., Ahire, S., Joshi, M., Ahirwar, R., & Joshi, A. N. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67744-6_14 · Cited by 8
Modern Touchscreen Keyboards as Intelligent User Interfaces: A Research Review
Zhai, S. (2017) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3025171.3026367 · Cited by 6
EGDE, A Soft Keyboard for Fast Typing for the Visually Challenged
Rajendran, C., Parab, C., & Gupta, S. (2016) · doi:10.1145/2851581.2890635 · Cited by 3
Research works printed but not linked: 6
- 10.5555/2305276.2305297 · doi:10.5555/2305276.2305297
- Tiago Guerreiro, Joaquim Jorge, and Daniel Gonçalves. 2012. Exploring the non-visual acquisition of targets on touch phones and tablets. In 2nd Workshop on Mobile Accessibility, MobileHCI.
- Mark S. Mayzner and Margaret Elizabeth Tresselt. 1965. Tables of single-letter and diagram frequency counts for various word-length and letter-position combinations. Psychonomic Monograph Supplements (1965).
- Kyle Montague, João Guerreiro, Hugo Nicolau, Tiago Guerreiro, André Rodrigues, and Daniel Gonçalves. 2016. Towards inviscid text-entry for blind people through non-visual word prediction interfaces. In CHI Workshop on Inviscid Text-Entry and Beyond.
- Sherry Ruan, Jacob O. Wobbrock, Kenny Liou, Andrew Ng, and James Landay. 2016. Speech is 3× faster than typing for English and Mandarin text entry on mobile devices. arXiv:1608.07323. · arXiv
- Crispin Thurlow and Alex Brown. 2003. Generation Txt? The sociolinguistics of young people’s text-messaging. Discourse Analysis Online 1, 1 (2003), 30.
Cited by
Komninos, A., Stefanis, V., & Garofalakis, J. (2023) · Multimodal Technologies and Interaction · doi:10.3390/mti7020022 · Cited by 16
Griggio, C. F., Gorman, B. M., & Tigwell, G. W. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641014 · Cited by 7
“Second, regarding communication, research has demonstrated the widely used QWERTY key-board layout is not optimised for use with a screen reader and the alternative layout of the keyboard letters can provide improved typing during text messaging [29].”
MethodYim, J., Ko, D., Kim, T., Kim, T., Hong, J., & Lee, G. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747801 · Cited by 4
“External devices like headphones or portable keyboards offer a potential solution, but it is inconvenient to carry and handle additional devices [17, 23].”
Exploring the impact of word prediction assistive features on smartphone keyboards for blind users
Alnfiai, M. M., & Kabir, M. A. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e36653 · Cited by 3
Signage Detection Based on Adaptive SIFT
Wang, J., Chen, S., Shao, J., Qu, H. Y., & Wang, Z. (2023) · Smart innovation, systems and technologies · doi:10.1007/978-981-99-6706-3_13
Metadata
- Authors
- Lottridge, D., Yoon, C., Burton, D., Wang, C., & Kaye, J.
- Affiliations
- University of Auckland (NZ)
- Yahoo (United States) (US)
- Mozilla Foundation (US)
- Venue
- ACM Transactions on Accessible Computing
- Published
- 2022-10-22
- Volume
- 15
- Issue
- 4
- Pages
- 1-22
- ISSN
- 1936-7228 (ISSN-L)
- 1936-7236
- OpenAlex
- W4225393491
- Cited by
- 6
- References
- 46
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 2 · 2023: 2
- FWCI
- 0.64
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Accessible Computing (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Keywords
- text messaging, blind users, virtual keyboard, text entry, assistive technology, smartphone accessibility, QWERTY keyboard, typing speed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.