Reviewing Speech Input with Audio: Differences between Blind and Sighted Users
The DOI matches its Crossref record, with a small difference in title or year; 56 of the 56 references it cites were found in a registry or an index.
Abstract
Speech input is a primary method of interaction for blind mobile device users, yet the process of dictating and reviewing recognized text through audio only (i.e., without access to visual feedback) has received little attention. A recent study found that sighted users could identify only about half of automatic speech recognition (ASR) errors when listening to text-to-speech output of the ASR results. Blind screen reader users, in contrast, may be better able to identify ASR errors through audio due to their greater use of speech interaction and increased ability to comprehend synthesized speech. To compare the experiences of blind and sighted users with speech input and ASR errors, as well as to compare their ability to identify ASR errors through audio-only interaction, we conducted a lab study with 12 blind and 12 sighted participants. The study included a semi-structured interview portion to qualitatively understand experiences with ASR, followed by a controlled speech input task to quantitatively compare participants’ ability to identify ASR errors in their dictated text. Findings revealed differences between blind and sighted participants in terms of how they use speech input and their level of concern for ASR errors (e.g., blind users were more highly concerned). In the speech input task, blind participants were able to identify only 40% of ASR errors, which, counter to our hypothesis, was not significantly different from sighted participants’ performance. In depth analysis of speech input, ASR errors, and strategy of identifying ASR errors scrutinized how participants entered a text with speech input and reviewed it. Our findings indicate the need for future work on how to support blind users in confidently using speech input to generate accurate, error-free text.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Some Experiments on the Recognition of Speech, with One and with Two Ears
Cherry, E. C. (1953) · The Journal of the Acoustical Society of America · doi:10.1121/1.1907229 · Cited by 4,549
psych: Procedures for Psychological, Psychometric, and Personality Research
Revelle, W. (2007) · CRAN: Contributed Packages · doi:10.32614/cran.package.psych · Cited by 2,965
The Media and Technology Usage and Attitudes Scale: An empirical investigation
Rosen, L. D., Whaling, K., Carrier, L. M., Cheever, N. A., & Rokkum, J. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2013.06.006 · Cited by 788
Local Standards for Sample Size at CHI
Caine, K. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858498 · Cited by 657
Pradhan, A., Mehta, K., & Findlater, L. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174033 · Cited by 448
Self-citationConfidence measures for speech recognition: A survey
Jiang, H. (2005) · Speech Communication · doi:10.1016/j.specom.2004.12.004 · Cited by 358
The microsoft 2016 conversational speech recognition system
Xiong, W., Droppo, J., Huang, X., Seide, F., Seltzer, M., Stolcke, A., Yu, D., & Zweig, G. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953159 · Cited by 326
Patterns of entry and correction in large vocabulary continuous speech recognition systems
Karat, C.-M., Halverson, C., Horn, D., & Karat, J. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303160 · Cited by 275
Tessa, a system to aid communication with deaf people
Cox, S., Lincoln, M., Tryggvason, J., Nakisa, M., Wells, M., Tutt, M., & Abbott, S. (2002) · Proceedings of the fifth international ACM conference on Assistive technologies · doi:10.1145/638249.638287 · Cited by 206
Multimodal error correction for speech user interfaces
Suhm, B., Myers, B., & Waibel, A. (2001) · ACM Transactions on Computer-Human Interaction · doi:10.1145/371127.371166 · Cited by 198 · cited 1×
“These visual editing approaches can be defined as unimodal (speech used to edit) and multi-modal (other input modalities used to edit) [58].”
BackgroundComparing Speech and Keyboard Text Entry for Short Messages in Two Languages on Touchscreen Phones
Ruan, S., Wobbrock, J. O., Liou, K., Ng, A., & Landay, J. A. (2018) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3161187 · Cited by 187
Automatic Speech Recognition Errors Detection and Correction: A Review
Errattahi, R., El Hannani, A., & Ouahmane, H. (2018) · Procedia Computer Science · doi:10.1016/j.procs.2018.03.005 · Cited by 155 · cited 12×
“We found that while both user groups confirmed the importance of speech input, blind participants used speech input more frequently than sighted participants (confirming results from Reference [3]).”
MethodInfluentialTools and Technologies for Computer-Aided Speech and Language Therapy
Saz, Ó., Yin, S.-C., Lleida, E., Rose, R. C., Vaquero, C., & Dueñas, W. R. R. (2009) · Speech Communication · doi:10.1016/j.specom.2009.04.006 · Cited by 153
Exploring the use of speech input by blind people on mobile devices
Azenkot, S., & Lee, N. B. (2013) · Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/2513383.2513440 · Cited by 151 · cited 10×
“We found that while both user groups confirmed the importance of speech input, blind participants used speech input more frequently than sighted participants (confirming results from Reference [3]).”
BackgroundMethodResultInfluentialVignovic, J. A., & Thompson, L. F. (2010) · Journal of Applied Psychology · doi:10.1037/a0018628 · Cited by 150
Current and future mobile and wearable device use by people with visual impairments
Ye, H., Malu, M., Oh, U., & Findlater, L. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557085 · Cited by 148
Self-citationCorbett, E., & Weber, A. (2016) · Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services · doi:10.1145/2935334.2935386 · Cited by 148
Predicting hyperarticulate speech during human-computer error resolution
Oviatt, S., MacEachern, M., & Levow, G.-A. (1998) · Speech Communication · doi:10.1016/s0167-6393(98)00005-3 · Cited by 144
Word Error Rate Estimation for Speech Recognition: e-WER
Ali, A. M. A., & Renals, S. J. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-2004 · Cited by 137
Effects of Spelling Errors on the Perception of Writers
Kreiner, D. S., Schnakenberg‐Ott, S. D., Green, A. G., Costello, M. J., & McClin, A. F. (2002) · The Journal of General Psychology · doi:10.1080/00221300209602029 · Cited by 129
A voice-input voice-output communication aid for people with severe speech impairment
Hawley, M. S., Cunningham, S. P., Green, P. D., Enderby, P., Palmer, R., Sehgal, S., & O'Neill, P. (2013) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2012.2209678 · Cited by 110
The vocal joystick: evaluation of voice-based cursor control techniques
Harada, S., Landay, J. A., Malkin, J., Li, X., & Bilmes, J. A. (2006) · Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/1168987.1169021 · Cited by 95
The vocal joystick: a voice-based human-computer interface for individuals with motor impairments
Bilmes, J. A., Dowden, P., Chizeck, H., Li, X., Malkin, J., Kilanski, K., Wright, R., Kirchhoff, K., Subramanya, A., Harada, S., et al. (2005) · Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT '05 · doi:10.3115/1220575.1220700 · Cited by 87
Understanding human management of automation errors
McBride, S. E., Rogers, W. A., & Fisk, A. D. (2014) · Theoretical Issues in Ergonomics Science · doi:10.1080/1463922x.2013.817625 · Cited by 82
JustSpeak: enabling universal voice control on Android
Zhong, Y., Raman, T. V., Burkhardt, C., Biadsy, F., & Bigham, J. P. (2014) · Proceedings of the 11th Web for All Conference · doi:10.1145/2596695.2596720 · Cited by 70
He, X., Deng, L., & Acero, A. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5947637 · Cited by 66
End-to-end Contextual Speech Recognition Using Class Language Models and a Token Passing Decoder
Chen, Z., Jain, M., Wang, Y.-Q., Seltzer, M. L., & Fuegen, C. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683573 · Cited by 63
Complementing Text Entry Evaluations with a Composition Task
Vertanen, K., & Kristensson, P. O. (2014) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2555691 · Cited by 62 · cited 1×
“The test prompts (in Appendix A) were selected from a list of short scenarios (“situations”) studied by Vertanen and Kristensson [62] for a freeform text composition task, such as: “Your housemate has been sick for the last week.”
MethodBerke, L., Caulfield, C., & Huenerfauth, M. (2017) · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3132525.3132541 · Cited by 62
Kafle, S., & Huenerfauth, M. (2017) · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3132525.3132542 · Cited by 59
The HearSay non-visual web browser
Borodin, Y., Mahmud, J., Ramakrishnan, I. V., & Stent, A. (2007) · Proceedings of the 2007 international cross-disciplinary conference on Web accessibility (W4A) - W4A '07 · doi:10.1145/1243441.1243444 · Cited by 53
A critical review and analysis on techniques of speech recognition: The road ahead
Haridas, A. V., Marimuthu, R., & Sivakumar, V. G. (2018) · International Journal of Knowledge-based and Intelligent Engineering Systems · doi:10.3233/kes-180374 · Cited by 49
Speech repair: quick error correction just by using selection operation for speech input interfaces
Ogata, J., & Goto, M. (2005) · Interspeech (USB) · doi:10.21437/interspeech.2005-86 · Cited by 45
A Large Inclusive Study of Human Listening Rates
Bragg, D., Bennett, C., Reinecke, K., & Ladner, R. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174018 · Cited by 43
Mihara, Y., Shibayama, E., & Takahashi, S. I. (2005) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1090785.1090801 · Cited by 37
On the intelligibility of fast synthesized speech for individuals with early-onset blindness
Stent, A., Syrdal, A., & Mishra, T. (2011) · The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/2049536.2049574 · Cited by 34
Capti-speak: a speech-enabled web screen reader
Ashok, V., Borodin, Y., Puzis, Y., & Ramakrishnan, I. V. (2015) · Proceedings of the 12th International Web for All Conference · doi:10.1145/2745555.2746660 · Cited by 33
The SRI speech understanding system
(1975) · doi:10.1109/tassp.1975.1162726 · Cited by 31
Scanning for Digital Content: How Blind and Sighted People Perceive Concurrent Speech
Guerreiro, J., & Gonçalves, D. J. V. (2016) · ACM Transactions on Accessible Computing · doi:10.1145/2822910 · Cited by 24
(2004) · doi:10.1097/00001610-200411000-00011 · Cited by 22
Identifying Speech Input Errors Through Audio-Only Interaction
Hong, J., & Findlater, L. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174141 · Cited by 22
Self-citationFischer, A. R. H., Price, K. J., & Sears, A. (2005) · International Journal of Human-Computer Interaction · doi:10.1207/s15327590ijhc1903_1 · Cited by 20
Automatic Assessment of Speech Capability Loss in Disordered Speech
Pellegrini, T., Fontan, L., Mauclair, J., Farinas, J., Alazard-Guiu, C., Robert, M., & Gatignol, P. (2015) · ACM Transactions on Accessible Computing · doi:10.1145/2739051 · Cited by 15
Seamless error correction interface for voice word processor
Choi, J., Kim, K., Lee, S., Kim, S., Lee, D., Lee, I., & Lee, G. G. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6289036 · Cited by 13
Error Correction of Speech Recognition by Custom Phonetic Alphabet Input for Ultra-Small Devices
Fujiwara, K. (2016) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2851581.2890380 · Cited by 13
Faster and Less Error-Prone: Supplementing an Accessible Keyboard with Speech Input
Bhikne, B., Joshi, A. N., Joshi, M., Jadhav, C., & Sakhardande, P. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-29381-9_18 · Cited by 11
Simple gesture-based error correction interface for smartphone speech recognition
Yuan, L., Iwano, K., & Shinoda, K. (2014) · Interspeech (USB) · doi:10.21437/interspeech.2014-302 · Cited by 11
How Much Faster Can You Type by Speaking in Hindi?
Bhikne, B., Joshi, A. N., Joshi, M., Ahire, S., & Maravi, N. (2018) · Indian Conference on Human-Computer Interaction (IndiaHCI) · doi:10.1145/3297121.3297123 · Cited by 9
Wang, L., Hu, T., Liu, P., et al. (2008) · Interspeech (USB) · doi:10.21437/interspeech.2008-659 · Cited by 9
Automatic speech recognition errors detection using supervised learning techniques
Errattahi, R., Hannani, A. E., Ouahmane, H., & Hain, T. (2016) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa.2016.7945669 · Cited by 6
Research works printed but not linked: 9
- W. Feng . 1994 . Using handwriting and gesture recognition to correct speech recognition errors . Urbana 51 (1994), 61801 . W. Feng. 1994. Using handwriting and gesture recognition to correct speech recognition errors. Urbana 51 (1994), 61801.
- Sahar Ghannay , Nathalie Camelin , and Yannick Esteve . 2015 . Which ASR errors are hard to detect . In Proceedings of the Errors by Humans and Machines in Multimedia, Multimodal and Multilingual Data Processing (ERRARE’15) Workshop, 11--13. Sahar Ghannay, Nathalie Camelin, and Yannick Esteve. 2015. Which ASR errors are hard to detect. In Proceedings of the Errors by Humans and Machines in Multimedia, Multimodal and Multilingual Data Processing (ERRARE’15) Workshop, 11--13.
- Sharon Goldwater , Dan Jurafsky , and Christopher D. Manning . 2008. Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase ASR error rates . In Proceedings of the Conference of the Association for Computational Linguistics and Human Language Technologies (ACL’08/ HLT’08) . 380--388. Sharon Goldwater, Dan Jurafsky, and Christopher D. Manning. 2008. Which words are hard to recognize? Prosodic, lexical, and disfluency factors that increase ASR error rates. In Proceedings of the Conference of the Association for Computational Linguistics and Human Language Technologies (ACL’08/ HLT’08). 380--388.
- Michael Gonchar . 2016. 650 Prompts for narrative and personal writing. New York Times 20 ( 2016 ). Michael Gonchar. 2016. 650 Prompts for narrative and personal writing. New York Times 20 (2016).
- 10.5555/1564144.1564149 · doi:10.5555/1564144.1564149
- 10.5555/646814.708131 · doi:10.5555/646814.708131
- Anja Moos and Jürgen Trouvain . 2007 . Comprehension of ultra-fast speech–blind vs.“normally hearing” persons . In Proceedings of the 16th International Congress of Phonetic Sciences. 677--680 . Anja Moos and Jürgen Trouvain. 2007. Comprehension of ultra-fast speech–blind vs.“normally hearing” persons. In Proceedings of the 16th International Congress of Phonetic Sciences. 677--680.
- Konstantinos Papadopoulos and Eleni Koustriava . 2015 . Comprehension of synthetic and natural speech: Differences among sighted and visually impaired young adults . In Proceedings of the International Conference on Enabling Access for Persons with Visual Impairment (ICEAPVI'15) . 147--151. Konstantinos Papadopoulos and Eleni Koustriava. 2015. Comprehension of synthetic and natural speech: Differences among sighted and visually impaired young adults. In Proceedings of the International Conference on Enabling Access for Persons with Visual Impairment (ICEAPVI'15). 147--151.
- Bhagath Parabattina and Pradip Das . 2004 . Acoustic phonetic approach for speech recognition: A review . Language 77 (2004), 93 . Bhagath Parabattina and Pradip Das. 2004. Acoustic phonetic approach for speech recognition: A review. Language 77 (2004), 93.
Cited by
MyMove: Facilitating Older Adults to Collect In-Situ Activity Labels on a Smartwatch with Speech
Kim, Y.-H., Chou, D., Lee, B., Danilovich, M., Lazar, A., Conroy, D. E., Kacorri, H., & Choe, E. K. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517457 · Cited by 39
Self-citationGamage, B., Do, T.-T., Price, N. S. C., Lowery, A., & Marriott, K. (2023) · The 25th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3597638.3608955 · Cited by 39
Defining Patterns for a Conversational Web
Pucci, E., Possaghi, I., Cutrupi, C. M., Báez, M., Cappiello, C., & Matera, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581145 · Cited by 34
Understanding How Blind Users Handle Object Recognition Errors: Strategies and Challenges
Hong, J., & Kacorri, H. (2024) · The 26th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663548.3675635 · Cited by 14
Self-citationHan, C., Mitra, P., & Billah, S. M. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642817 · Cited by 26
Metadata
- Authors
- Hong, J., Vaing, C., Kacorri, H., & Findlater, L.
- Affiliations
- University of Maryland, College Park (US)
- University of Washington (US)
- Venue
- ACM Transactions on Accessible Computing
- Published
- 2020-03-31
- Volume
- 13
- Issue
- 1
- Pages
- 1-28
- ISSN
- 1936-7228 (ISSN-L)
- 1936-7236
- OpenAlex
- W3019547849
- Cited by
- 11
- References
- 56
- Citations by year
- 2025: 5 · 2024: 3 · 2023: 2 · 2022: 1
- FWCI
- 1.35
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Accessible Computing (publishedVersion)
- Topics
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Keywords
- speech input, speech recognition errors, blind users, screen readers, text-to-speech, error identification, mobile accessibility, speech recognition accuracy, user experience
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.