Reviewing Speech Input with Audio: Differences between Blind and Sighted Users
The DOI matches its Crossref record, with a small difference in title or year; 56 of the 56 references it cites were found in a registry or an index.
Abstract
Speech input is a primary method of interaction for blind mobile device users, yet the process of dictating and reviewing recognized text through audio only (i.e., without access to visual feedback) has received little attention. A recent study found that sighted users could identify only about half of automatic speech recognition (ASR) errors when listening to text-to-speech output of the ASR results. Blind screen reader users, in contrast, may be better able to identify ASR errors through audio due to their greater use of speech interaction and increased ability to comprehend synthesized speech. To compare the experiences of blind and sighted users with speech input and ASR errors, as well as to compare their ability to identify ASR errors through audio-only interaction, we conducted a lab study with 12 blind and 12 sighted participants. The study included a semi-structured interview portion to qualitatively understand experiences with ASR, followed by a controlled speech input task to quantitatively compare participants’ ability to identify ASR errors in their dictated text. Findings revealed differences between blind and sighted participants in terms of how they use speech input and their level of concern for ASR errors (e.g., blind users were more highly concerned). In the speech input task, blind participants were able to identify only 40% of ASR errors, which, counter to our hypothesis, was not significantly different from sighted participants’ performance. In depth analysis of speech input, ASR errors, and strategy of identifying ASR errors scrutinized how participants entered a text with speech input and reviewed it. Our findings indicate the need for future work on how to support blind users in confidently using speech input to generate accurate, error-free text.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Some Experiments on the Recognition of Speech, with One and with Two Ears
Cherry, E. C. (1953) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907229 · Cited by 4,549
psych: Procedures for Psychological, Psychometric, and Personality Research
Revelle, W. (2007) · CRAN: Contributed Packages · doi:10.32614/cran.package.psych · Cited by 2,965
The Media and Technology Usage and Attitudes Scale: An empirical investigation
Rosen, L. D., Whaling, K., Carrier, L. M., Cheever, N. A., & Rokkum, J. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2013.06.006 · Cited by 788
Local Standards for Sample Size at CHI
Caine, K. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858498 · Cited by 657
Pradhan, A., Mehta, K., & Findlater, L. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174033 · Cited by 448
Self-citationConfidence measures for speech recognition: A survey
Jiang, H. (2005) · Speech Communication · doi:10.1016/j.specom.2004.12.004 · Cited by 358
The microsoft 2016 conversational speech recognition system
Xiong, W., Droppo, J., Huang, X., Seide, F., Seltzer, M., Stolcke, A., Yu, D., & Zweig, G. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953159 · Cited by 326
Patterns of entry and correction in large vocabulary continuous speech recognition systems
Karat, C.-M., Halverson, C., Horn, D., & Karat, J. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303160 · Cited by 275
Research works printed but not linked: 9
- W. Feng . 1994 . Using handwriting and gesture recognition to correct speech recognition errors . Urbana 51 (1994), 61801 . W. Feng. 1994. Using handwriting and gesture recognition to correct speech recognition errors. Urbana 51 (1994), 61801.
- Sahar Ghannay , Nathalie Camelin , and Yannick Esteve . 2015 . Which ASR errors are hard to detect . In Proceedings of the Errors by Humans and Machines in Multimedia, Multimodal and Multilingual Data Processing (ERRARE’15) Workshop, 11--13. Sahar Ghannay, Nathalie Camelin, and Yannick Esteve. 2015. Which ASR errors are hard to detect. In Proceedings of the Errors by Humans and Machines in Multimedia, Multimodal and Multilingual Data Processing (ERRARE’15) Workshop, 11--13.
- Sharon Goldwater , Dan Jurafsky , and Christopher D. Manning . 2008. Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase ASR error rates . In Proceedings of the Conference of the Association for Computational Linguistics and Human Language Technologies (ACL’08/ HLT’08) . 380--388. Sharon Goldwater, Dan Jurafsky, and Christopher D. Manning. 2008. Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase ASR error rates. In Proceedings of the Conference of the Association for Computational Linguistics and Human Language Technologies (ACL’08/ HLT’08). 380--388.
- Michael Gonchar . 2016. 650 Prompts for narrative and personal writing. New York Times 20 ( 2016 ). Michael Gonchar. 2016. 650 Prompts for narrative and personal writing. New York Times 20 (2016).
- 10.5555/1564144.1564149 · doi:10.5555/1564144.1564149
- 10.5555/646814.708131 · doi:10.5555/646814.708131
- Anja Moos and Jürgen Trouvain . 2007 . Comprehension of ultra-fast speech–blind vs.“normally hearing” persons . In Proceedings of the 16th International Congress of Phonetic Sciences. 677--680 . Anja Moos and Jürgen Trouvain. 2007. Comprehension of ultra-fast speech–blind vs.“normally hearing” persons. In Proceedings of the 16th International Congress of Phonetic Sciences. 677--680.
- Konstantinos Papadopoulos and Eleni Koustriava . 2015 . Comprehension of synthetic and natural speech: Differences among sighted and visually impaired young adults . In Proceedings of the International Conference on Enabling Access for Persons with Visual Impairment (ICEAPVI'15) . 147--151. Konstantinos Papadopoulos and Eleni Koustriava. 2015. Comprehension of synthetic and natural speech: Differences among sighted and visually impaired young adults. In Proceedings of the International Conference on Enabling Access for Persons with Visual Impairment (ICEAPVI'15). 147--151.
- Bhagath Parabattina and Pradip Das . 2004 . Acoustic phonetic approach for speech recognition: A review . Language 77 (2004), 93 . Bhagath Parabattina and Pradip Das. 2004. Acoustic phonetic approach for speech recognition: A review. Language 77 (2004), 93.
Cited by
MyMove: Facilitating Older Adults to Collect In-Situ Activity Labels on a Smartwatch with Speech
Kim, Y.-H., Chou, D., Lee, B., Danilovich, M., Lazar, A., Conroy, D. E., Kacorri, H., & Choe, E. K. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517457 · Cited by 39
Self-citationGamage, B., Do, T.-T., Price, N. S. C., Lowery, A., & Marriott, K. (2023) · The 25th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3597638.3608955 · Cited by 39
Defining Patterns for a Conversational Web
Pucci, E., Possaghi, I., Cutrupi, C. M., Báez, M., Cappiello, C., & Matera, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581145 · Cited by 34
Understanding How Blind Users Handle Object Recognition Errors: Strategies and Challenges
Hong, J., & Kacorri, H. (2024) · The 26th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663548.3675635 · Cited by 14
Self-citationHan, C., Mitra, P., & Billah, S. M. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642817 · Cited by 26
Limitations in speech recognition for young adults with down syndrome
Cibrian, F. L., Chen, Y. ‘., Anderson, K., Abrahamsson, C. M., & Motti, V. G. (2025) · Universal Access in the Information Society · doi:10.1007/s10209-025-01197-4 · Cited by 13
Hennekeuser, D., Vaziri, D. D., Golchinfar, D., & Stevens, G. (2024) · Interacting with Computers · doi:10.1093/iwc/iwae018 · Cited by 9
GestureVoice: Enabling Multimodal Text Editing for Blind Users Using Gestures and Voice
Khanna, P., Reddy, M. P., Ramakrishnan, I. V., Bi, X., & Balasubramanian, A. (2025) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663547.3746388 · Cited by 2
Using Confidence Scores to Improve Eyes-free Detection of Speech Recognition Errors
Nowrin, S., & Vertanen, K. (2025) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/3733155.3734896
Could You Hear That? Identifying Marathi Phrases Suitable for Aural Transcription Tasks
Shetye, S. A., & Joshi, A. N. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-80829-6_4
Metadata
- Authors
- Hong, J., Vaing, C., Kacorri, H., & Findlater, L.
- Affiliations
- University of Maryland, College Park (US)
- University of Washington (US)
- Venue
- ACM Transactions on Accessible Computing
- Published
- 2020-03-31
- Volume
- 13
- Issue
- 1
- Pages
- 1-28
- ISSN
- 1936-7228 (ISSN-L)
- 1936-7236
- OpenAlex
- W3019547849
- Cited by
- 11
- References
- 56
- Citations by year
- 2025: 5 · 2024: 3 · 2023: 2 · 2022: 1
- FWCI
- 1.35
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Accessible Computing (publishedVersion)
- Topics
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Keywords
- speech input, speech recognition errors, blind users, screen readers, text-to-speech, error identification, mobile accessibility, speech recognition accuracy, user experience
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.