Bridging Screen Readers and Voice Assistants for Enhanced Eyes-Free Web Search
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
People with visual impairments often rely on screen readers when interacting with computer systems. Increasingly, these individuals also make extensive use of voice-based virtual assistants (VAs). We conducted a survey of 53 people who are legally blind to identify the strengths and weaknesses of both technologies, as well as the unmet opportunities at their intersection. We learned that virtual assistants are convenient and accessible, but lack the ability to deeply engage with content (e.g., read beyond the first few sentences of Wikipedia), and the ability to get a quick overview of the landscape (list alternative search results & suggestions). In contrast, screen readers allow for deep engagement with content (when content is accessible), and provide fine-grained navigation & control, but at the cost of increased complexity, and reduced walk-up-and-use convenience. In this demonstration, we showcase VERSE, a system that combines the positive aspects of VAs and screen readers, and allows other devices (e.g., smart watches) to serve as optional input accelerators. Together, these features allow people with visual impairments to deeply engage with web content through voice interaction.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Blind users particularly desire the ability to perform complex tasks with finer control than is currently possible through commercial voice interfaces [1,37].”
From Reading Between the Guidelines: How Commercial Voice Assistant Guidelines Hinder Accessibility for Blind Users (2019) · cites this paper as [37]
In the citing paper
“These studies primarily observe product reviews, user testimony, or in situ use of VAPAs.”
From Reading Between the Guidelines: How Commercial Voice Assistant Guidelines Hinder Accessibility for Blind Users (2019) · cites this paper as [37]
Citations
Cites
Luger, E., & Sellen, A. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858288 · Cited by 1,176
(2017) · Research Methods in Human Computer Interaction · doi:10.1016/b978-0-12-805390-4.09991-x · Cited by 889
Exploratory Search: Beyond the Query-Response Paradigm
White, R. W., & Roth, R. A. (2009) · Synthesis lectures on information concepts, retrieval, and services · doi:10.2200/s00174ed1v01y200901icr003 · Cited by 632
Self-citationPradhan, A., Mehta, K., & Findlater, L. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174033 · Cited by 448
Self-citationAbdolrahmani, A., Kuber, R., & Branham, S. M. (2018) · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3234695.3236344 · Cited by 172 · cited 3×
“The desire for customization of VAs was mentioned by only one participant in our survey, but has been identified as a limitation of VAs in prior work [1].”
BackgroundMethodInfluential
Research works printed but not linked: 4
- Jonathan Lazar , Jinjuan Heidi Feng, and Harry Hochheiser . 2017 . Research methods in human-computer interaction. Morgan Kaufmann . Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Research methods in human-computer interaction. Morgan Kaufmann.
- Rishabh Mehrotra , A Hassan Awadallah , Ahmed E Kholy , and Imed Zitouni . 2017 . Hey Cortana! Exploring the use cases of a Desktop based Digital Assistant . In SIGIR 1st International Workshop on Conversational Approaches to Information Retrieval (CAIR'17) . Rishabh Mehrotra, A Hassan Awadallah, Ahmed E Kholy, and Imed Zitouni. 2017. Hey Cortana! Exploring the use cases of a Desktop based Digital Assistant. In SIGIR 1st International Workshop on Conversational Approaches to Information Retrieval (CAIR'17).
- Janice Y. Tsai Tawfiq Ammari Abraham Wallin and Jofish Kaye. 2018. Alexa play some music: Categorization of Alexa Commands. In Voice-based Conversational UX Studies and Design Wokrshop at CHI. ACM. Janice Y. Tsai Tawfiq Ammari Abraham Wallin and Jofish Kaye. 2018. Alexa play some music: Categorization of Alexa Commands. In Voice-based Conversational UX Studies and Design Wokrshop at CHI. ACM.
- Alexandra Vtyurina , Adam Fourney , Meredith Ringel Morris , Leah Findlater, and Ryen W. White. 2019 . Demo video of VERSE. https://aka.ms/verse_demo_www2019 Alexandra Vtyurina, Adam Fourney, Meredith Ringel Morris, Leah Findlater, and Ryen W. White. 2019. Demo video of VERSE. https://aka.ms/verse_demo_www2019
Cited by
Das, M., Gergle, D., & Piper, A. M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359293 · Cited by 108
“…have improved the accessibility of web-based content by developing various assistive technologies (e.g., auditory [38, 73], speech-controlled [9, 86], haptic [16, 48] and spatial [44] interactions), personalized and opportunistic adaptations [8, 25, 37, 100], semantic web modeling [8, 9, 16],…”
BackgroundAn Empirical Study of Older Adult’s Voice Assistant Use for Health Information Seeking
Brewer, R., Pierce, C., Upadhyay, P., & Park, L. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3484507 · Cited by 89
Browsing with Alexa: interrogating the impact of voice assistants as web interfaces
Natale, S., & Cooke, H. (2021) · Media, Culture & Society · doi:10.1177/0163443720983295 · Cited by 76
“…greet these technologies as a new epoch for human-computer interaction and for their potential benefits in regard to issues such as accessibility (Vtyurina et al., 2019), critics highlight that they produce or reproduce stereotypes of gender and race (Schiller and McMahon, 2019) and raise…”
BackgroundBranham, S. M., & Mukkath Roy, A. R. (2019) · Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3308561.3353797 · Cited by 75
“Numerous recent research efforts have centered on the accessibility constraints and potential of VAPAs to serve older adults [28,38,40] and people with disabilities [1,3,7,29,37].”
BackgroundInfluentialAbdolrahmani, A., Storer, K. M., Roy, A. R. M., Kuber, R., & Branham, S. M. (2019) · ACM Transactions on Accessible Computing · doi:10.1145/3368426 · Cited by 68
Seven HCI Grand Challenges Revisited: Five-Year Progress
Stephanidis, C., Salvendy, G., Antona, M., Duffy, V. G., Gao, Q., Karwowski, W., Konomi, S., Nah, F. F., Ntoa, S., Rau, P. P., et al. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2450411 · Cited by 69
“…facilitating communication with their care partners (Zubatiy et al., 2021), as well as persons with visual impairments by facilitating their interaction with a wide spectrum of technologies, opening up avenues that would otherwise be inaccessible (Abdolrahmani et al., 2018; Vtyurina et al., 2019).”
Effectiveness of Using Voice Assistants in Learning: A Study at the Time of COVID-19
Sáiz-Manzanares, M. C., Marticorena-Sánchez, R., & Ochoa-Orihuel, J. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17155618 · Cited by 47
Firefox Voice: An Open and Extensible Voice Assistant Built Upon the Web
Cambre, J., Williams, A. C., Razi, A., Bicking, I., Wallin, A., Tsai, J., Kulkarni, C., & Kaye, J. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445409 · Cited by 39
Ngueajio, M. K., & Washington, G. J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-21707-4_30 · Cited by 114
“A recent study [53] involving legally blind adults has for objectives to identify the pros and cons of virtual assistants (VA) and screen readers from their viewpoint, address and suggest possible design intersectionality that can be leveraged to improve ASR systems accessibility for these users.”
Background"All in the Same Boat": Tradeoffs of Voice Assistant Ownership for Mixed-Visual-Ability Families
Storer, K. M., Judge, T. K., & Branham, S. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376225 · Cited by 35
Rong, E. Z., Zhou, M. M., Lu, Z., & Fan, M. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533514 · Cited by 41
A Dataset for Interactive Vision-Language Navigation with Unknown Command Feasibility
Burns, A., Arsan, D., Agrawal, S., Kumar, R., Saenko, K., & Plummer, B. A. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20074-8_18 · Cited by 32
Defining Patterns for a Conversational Web
Pucci, E., Possaghi, I., Cutrupi, C. M., Báez, M., Cappiello, C., & Matera, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581145 · Cited by 34
“Also in line with previous work [56], this suggests that the next studies should put efort into discovering new synergies among existing assistive technologies (such as screen readers) and the conversational paradigm, to give the users a chance to choose the most adequate one depending on their…”
BackgroundInfluentialAbdolrahmani, A., Gupta, M. H., Vader, M.-L., Kuber, R., & Branham, S. M. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445638 · Cited by 30
Entity Recommendation for Everyday Digital Tasks
Jacucci, G., Daee, P., Vuong, T., Andolina, S., Klouche, K., Sjöberg, M., Ruotsalo, T., & Samuel Kaski, A. (2021) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3458919 · Cited by 27
“Traditional approaches to capturing text are based on Accessibility Application programming interface (APIs) [78, 99].”
MethodZimmerman, J., Steinfeld, A. M., Tomasic, A., & Romero, O. J. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517789 · Cited by 32
“In a survey probing why people might choose to use a screen-reader to access the web over a smart speaker, researchers found [81]:”
BackgroundAqle, A., Al-Thani, D., & Jaoua, A. (2022) · Universal Access in the Information Society · doi:10.1007/s10209-020-00777-w · Cited by 16
“A recent study has shown that voice-based VAs can improve the navigation systems [81].”
BackgroundUsing Phoneme Representations to Build Predictive Models Robust to ASR Errors
Fang, A., Filice, S., Limsopatham, N., & Rokhlenko, O. (2020) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3397271.3401050 · Cited by 18
“voice search engine [35] and voice shopping [14].”
BackgroundTiwary, T., & Mahapatra, R. P. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-14494-y · Cited by 16
“Web access plays a vital role in everyone ’ s life and offers various applications such as education, work, money transfer, and online shopping [1].”
BackgroundYücel, Y., & Rızvanoğlu, K. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103403 · Cited by 15
Van, B. P., Wong, P. S. P., & Abbasnejad, B. (2025) · Journal of Information Technology in Construction · doi:10.36680/j.itcon.2025.011 · Cited by 41
Comparing Non-Visual and Visual Information Foraging on The Web
Upadhyay, P. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3383025 · Cited by 14
Jiang, C., Lei, W., Kuang, E., Han, T., & Fan, M. (2023) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3597638.3608423 · Cited by 9
Yu, Y., Ryskeldiev, B., Tsutsui, A., Gillingham, M., & Wang, Y. (2025) · Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663547.3746353 · Cited by 8
The Perturbing Mediatization of Voice-based Virtual Assistants: The Case of Alexa
Fortunati, L., Edwards, A. P., & Edwards, C. C. (2024) · Human-Machine Communication · doi:10.30658/hmc.7.6 · Cited by 4
Hennekeuser, D., Vaziri, D. D., Golchinfar, D., & Stevens, G. (2024) · Interacting with Computers · doi:10.1093/iwc/iwae018 · Cited by 9
Alzalabny, S., Moured, O., Müller, K., Schwarz, T., Rapp, B. E., & Stiefelhagen, R. (2024) · Multimodal Technologies and Interaction · doi:10.3390/mti8110102 · Cited by 5
Understanding Voice Search Behavior: Review and Synthesis of Research
Xing, Z., Yuan, X., Wu, D., Huang, Y., & Mostafa, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60117-1_23 · Cited by 10
Xie, I., Choi, W., Lee, H. S., Hong, B. H., & Wang, S. (2025) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24997 · Cited by 6
“Therefore, it is important to provide additional input features beyond screen readers to enhance eyes-free and hands-free web searches (Vtyurina et al., 2019).”
A Suite of Generative Tasks for Multi-Level Multimodal Webpage Understanding
Burns, A., Srinivasan, K., Ainslie, J., Brown, G., Plummer, B. A., Saenko, K., Ni, J., & Guo, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.119 · Cited by 27
Referring to Screen Texts with Voice Assistants
Bhargava, S., Dhoot, A., Jonsson, I.-M., Nguyen, H. L., Patel, A., Yu, H., & Renkens, V. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-industry.72 · Cited by 3
Han, X., & YEH, T. P. (2020) · Cited by 5
Improving a Conversational Speech Recognition System Using Phonetic and Neural Transcript Correction
Campos-Soberanis, M., Campos-Sobrino, D., & Viana-Cámara, R. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-89820-5_4 · Cited by 2
Megavath, R., Indra, G., Al‐Rasheed, A., Alqahtani, M. S., Abbas, M. A. E., Almohiy, H., Jambi, L. K., & Soufiene, B. O. (2023) · Research Square · doi:10.21203/rs.3.rs-2814782/v1 · Cited by 3
Acceptance of Job Access with Speech (Jaws) as an Assistive Computer Application Software
Ndlovu, L., Bayaga, A., & Blignaut, S. (2023) · Studies in computational intelligence · doi:10.1007/978-3-031-33309-5_25 · Cited by 2
Nayak, A. K., Prakash, Y., Jayarathna, S., Lee, H.-N., & Ashok, V. G. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791509 · Cited by 2
“…individuals adopt and use various technologies primarily designed for sighted users (e.g., smartphones [14, 61, 101], virtual assistants [4, 119, 125, 196], and virtual reality technology [53, 87]), with ‘add-on’ accessibility features (e.g., SR and SM support) to accommodate disabled individuals.”
Choi, S., Seo, J., Krishnan, A., Kamath, S., Hernandez, M. E., & Kitsiou, S. (2026) · Disability and Rehabilitation: Assistive Technology · doi:10.1080/17483107.2026.2663134
“Recent research also has shown that BlV individuals, particularly those who are proficient screen reader users, have developed sophisticated touch-based interaction strategies that they often prefer over voice interactions [18].”
Speech-Based Web Navigation for Limited Mobility Users.
Radostev, V., Berger, S., Tabrizi, J., Kamyshev, P., & Suzuki, H. (2019) · Interspeech (USB)
Megayanti, A. (2020) · FLYWHEEL Jurnal Teknik Mesin Untirta · doi:10.36055/fwl.v2i1.7497
Application of Voice Personal Assistants in the Context of Smart University
Ochoa-Orihuel, J. (2022) · Repositorio Institucional de la Universidad de Burgos (University of Burgos) · doi:10.36443/10259/7454
Using Confidence Scores to Improve Eyes-free Detection of Speech Recognition Errors
Nowrin, S., & Vertanen, K. (2025) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/3733155.3734896
“In recent years, there have been notable advancements in Automatic Speech Recognition (ASR) technology, enabling eyes-free interaction [8, 10, 15, 29] and improving accessibility for devices without a visual display (e.g. Amazon Echo, Google Home).”
Exploring perspectives of teachers of students with visual impairments on teaching screen readers
Kırboyun, S. (2026) · Assistive Technology · doi:10.1080/10400435.2026.2630935 · Cited by 1
“While previous research shows screen readers benefit adults with VI (Argyropoulos et al., 2018; Douglas, 2001; Kelly, 2011; Samuels, 2008; Vtyurina et al., 2019; Zhou et al., 2012), there is insufficient evidence to suggest similar outcomes for younger students.”
Text and Voice-augmented AI Chatbots: The Role of Input Modality in Conversational CommercePreprint
Bojd, B., Aravindakshan, A., & Rutz, O. J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6985878
Forootan, A., Barter, D., & Coppin, P. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29656-6_18
Metadata
- Authors
- Vtyurina, A., Fourney, A., Morris, M. R., Findlater, L., & White, R. W.
- Affiliations
- University of Waterloo (CA)
- Microsoft (United States) (US)
- University of Washington (US)
- Venue
- The World Wide Web Conference
- Publisher
- ACM
- Published
- 2019-05-13
- Pages
- 3590-3594
- OpenAlex
- W2945524287
- Cited by
- 42
- References
- 5
- Citations by year
- 2026: 5 · 2025: 4 · 2024: 4 · 2023: 7 · 2022: 6 · 2021: 4 · 2020: 9 · 2019: 3
- FWCI
- 10.57
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International World Wide Web Conference (WWW) (publishedVersion, cc-by)
- Topics
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Keywords
- screen readers, voice assistants, visual impairment, blind users, web search, voice interaction, smartwatches, poetry, assistive technology, web accessibility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.