Auto-Suggesting Browsing Actions for Personalized Web Screen Reading
Open accessThe DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
Web browsing has never been easy for blind people, primarily due to the serial press-and-listen interaction mode of screen readers -- their "go-to'' assistive technology. Even simple navigational browsing actions on a page require a multitude of shortcuts. Auto-suggesting the next browsing action has the potential to assist blind users in swiftly completing various tasks with minimal effort. Extant auto-suggest feature in web pages is limited to filling form fields; in this paper, we generalize it to any web screen-reading browsing action, e.g., navigation, selection, etc. Towards that, we introduce SuggestOmatic, a personalized and scalable unsupervised approach for predicting the most likely next browsing action of the user, and proactively suggesting it to the user so that the user can avoid pressing a lot of shortcuts to complete that action. SuggestOmatic rests on two key ideas. First, it exploits the user's Action History to identify and suggest a small set of browsing actions that will, with high likelihood, contain an action which the user will want to do next, and the chosen action is executed automatically. Second, the Action History is represented as an abstract temporal sequence of operations over semantic web entities called Logical Segments - a collection of related HTML elements, e.g., widgets, search results, menus, forms, etc.; this semantics-based abstract representation of browsing actions in the Action History makes SuggestOmatic scalable across websites, i.e., actions recorded in one website can be used to make suggestions for other similar websites. We also describe an interface that uses an off-the-shelf physical Dial as an input device that enables SuggestOmatic to work with any screen reader. The results of a user study with 12 blind participants indicate that SuggestOmatic can significantly reduce the browsing task times by as much as 29% when compared with a hand-crafted macro-based web automation solution.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
What Frustrates Screen Reader Users on the Web: A Study of 100 Blind Users
Lazar, J., Allen, A., Kleinman, J., & Malarkey, C. (2007) · International Journal of Human-Computer Interaction · doi:10.1080/10447310709336964 · Cited by 354 · cited 1×
“However, this serial “press-and-listen” interface of screen readers have been shown to be inadequate for providing a satisfactory and usable web-browsing experience for blind users ([12, 18]).”
BackgroundMore than meets the eye: a survey of screen-reader browsing strategies
Borodin, Y., Bigham, J. P., Dausch, G., & Ramakrishnan, I. V. (2010) · Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1805986.1806005 · Cited by 222
Self-citation
Cited by
Das, M., Gergle, D., & Piper, A. M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359293 · Cited by 108
“, auditory [38, 73], speech-controlled [9, 86], haptic [16, 48] and spatial [44] interactions), personalized and opportunistic adaptations [8, 25, 37, 100], semantic web modeling [8, 9, 16], shared browsing [68], and collaborative [40, 78] and automated [35] evaluation techniques.”
BackgroundDas, M., Piper, A. M., & Gergle, D. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3480169 · Cited by 32
Barbosa, N. M., Hayes, J., Kaushik, S., & Wang, Y. (2022) · ACM Transactions on Accessible Computing · doi:10.1145/3519032 · Cited by 28
“For example, the ability to interact via commands has been shown to be a promising modality for users with visual impairments, such as in accessing key sections of a given web page [28] and performing the next action on a given web page, such as when booking flights [7], and interacting with virtual world games [26].”
BackgroundMethodInSupport: Proxy Interface for Enabling Efficient Non-Visual Interaction with Web Data Records
Ferdous, J., Lee, H.-N., Jayarathna, S., & Ashok, V. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511126 · Cited by 11
Self-citationFerdous, J., Lee, H.-N., Jayarathna, S., & Ashok, V. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3579364 · Cited by 7
Self-citation
Metadata
- Authors
- Ashok, V., Billah, S. M., Borodin, Y., & Ramakrishnan, IV
- Venue
- Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization
- Publisher
- ACM
- Published
- 2019-06-07
- Volume
- 2019
- Pages
- 252-260
- OpenAlex
- W2954579733
- Cited by
- 19
- References
- 2
- FWCI
- 3.39
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Auto-Suggesting Browsing Actions for Personalized Web Screen Reading. https://app.cochat.ai/papers/10.1145/3320435.3320460
- The paper
- Ashok, V., Billah, S. M., Borodin, Y., & Ramakrishnan, IV. (2019). Auto-Suggesting Browsing Actions for Personalized Web Screen Reading. Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization. https://doi.org/10.1145/3320435.3320460
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.