Interaction Proxies for Runtime Repair and Enhancement of Mobile Application Accessibility
The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
Abstract
We introduce interaction proxies as a strategy for runtime repair and enhancement of the accessibility of mobile applications. Conceptually, interaction proxies are inserted between an application's original interface and the manifest interface that a person uses to perceive and manipulate the application. This strategy allows third-party developers and researchers to modify an interaction without an application's source code, without rooting the phone, without otherwise modifying an application, while retaining all capabilities of the system (e.g., Android's full implementation of the TalkBack screen reader). This paper introduces interaction proxies, defines a design space of interaction re-mappings, identifies necessary implementation abstractions, presents details of implementing those abstractions in Android, and demonstrates a set of Android implementations of interaction proxies from throughout our design space. We then present a set of interviews with blind and low-vision people interacting with our prototype interaction proxies, using these interviews to explore the seamlessness of interaction, the perceived usefulness and potential of interaction proxies, and visions of how such enhancements could gain broad usage. By allowing third-party developers and researchers to improve an interaction, interaction proxies offer a new approach to personalizing mobile application accessibility and a new approach to catalyzing development, deployment, and evaluation of mobile accessibility enhancements.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In this paper, we describe our implementation which, although it leveraged concepts from Interaction Proxies, was entirely novel.”
From Interactiles: 3D Printed Tactile Interfaces to Enhance Mobile Touchscreen Accessibility (2018) · cites this paper as [32]
Citations
Cites
del Carmen Malbrán, M. (2011) · Journal of Policy and Practice in Intellectual Disabilities · doi:10.1111/j.1741-1130.2011.00320.x · Cited by 1,674
Direct Manipulation Interfaces
Hutchins, E. L., Hollan, J. D., & Norman, D. A. (1985) · Human–Computer Interaction · doi:10.1207/s15327051hci0104_2 · Cited by 1,307
Past, present, and future of user interface software tools
Myers, B., Hudson, S. E., & Pausch, R. (2000) · ACM Transactions on Computer-Human Interaction · doi:10.1145/344949.344959 · Cited by 808 · cited 1×
“Although such personalization is promising for many accessibility needs, adoption of such methods is limited by a need for interfaces to be re-written as abstract specifications [45].”
BackgroundVizWiz: nearly real-time answers to visual questions
Bigham, J. P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R. C., Miller, R., Tatarowicz, A., White, B., White, S., et al. (2010) · Proceedings of the 23nd annual ACM symposium on User interface software and technology · doi:10.1145/1866029.1866080 · Cited by 601 · cited 1×
“VizWiz examines supporting blind people in using a mobile device to take a picture and ask a crowd a question about that picture [5,11].”
BackgroundAbility-Based Design: Concept, Principles and Examples
Wobbrock, J. O., Kane, S. K., Gajos, K. Z., Harada, S., & Froehlich, J. (2011) · ACM Transactions on Accessible Computing · doi:10.1145/1952383.1952384 · Cited by 567 · cited 5×
“Figure 10 shows an example of a personalized interface layout, motivated by SUPPLE’s approach to arranging entire interfaces to match an individual’s motor abilities [23,58].”
BackgroundMethodInfluentialSelf-citationVizWiz: nearly real-time answers to visual questions
Bigham, J. P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R. C., Tatarowicz, A., White, B., White, S., & Yeh, T. (2010) · Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1805986.1806020 · Cited by 536
Kane, S. K., Jayant, C., Wobbrock, J. O., & Ladner, R. E. (2009) · Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/1639642.1639663 · Cited by 383 · cited 1×
“Prior research examines how people adapt to accessibility barriers in mobile devices and how devices can enhance access in the physical world [35].”
BackgroundSelf-citationMaking computers easier for older adults to use: area cursors and sticky icons
Worden, A., Walker, N., Bharat, K., & Hudson, S. (1997) · Proceedings of the ACM SIGCHI Conference on Human factors in computing systems · doi:10.1145/258549.258724 · Cited by 376 · cited 1×
“Examples include making targets “sticky” [29,59], dynamically modifying cursor behavior according to a person’s movement profile [57], and breaking a pointing interaction into multiple interactions that disambiguate the intended target [22,33,63].”
BackgroundKane, S. K., Bigham, J. P., & Wobbrock, J. O. (2008) · Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/1414471.1414487 · Cited by 374 · cited 3×
“, Slide Rule [34], Android’s TalkBack, iOS’s VoiceOver), which re-maps touch to support navigation separate from activation and may completely disable visual display of an app.”
BackgroundInfluentialSelf-citationEnsuring Digital Accessibility Through Process and Policy
Lazar, J., Goldstein, D., & Taylor, A. (2015) · Elsevier eBooks · doi:10.1016/c2013-0-13367-3 · Cited by 359 · cited 1×
“, Android’s TalkBack, iOS’s VoiceOver), and now improve accessibility of apps on those platforms [42].”
Background
Cited by
SeeingVR: A Set of Tools to Make Virtual Reality More Accessible to People with Low Vision
Zhao, Y., Cutrell, E., Holz, C., Morris, M. R., Ofek, E., & Wilson, A. D. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300341 · Cited by 207
“introduced interaction proxies [81], which improved the accessibility of mobile apps by adding an overlay that modified the input and output of the underlying app.”
BackgroundScreen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels
Zhang, X., de Greef, L., Swearngin, A., White, S., Murray, K., Yu, L., Shan, Q., Nichols, J., Wu, J., Fleizach, C., et al. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445186 · Cited by 144
“tory of such information for runtime repair [80, 81].”
BackgroundSelf-citationLi, T. J.-J., Radensky, M., Jia, J., Singarajah, K., Mitchell, T. M., & Myers, B. A. (2019) · Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3332165.3347899 · Cited by 111
“P UMICE uses an interaction proxy overlay [53] for recording, so that it can record all values visible on the screen, not limited to the selectable or clickable ones.”
Unblind your apps: predicting natural-language labels for mobile GUI components by deep learning
Chen, J., Chen, C., Xing, Z., Xu, X., Zhu, L., Li, G., & Wang, J. (2020) · Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering · doi:10.1145/3377811.3380327 · Cited by 109
“For other accessibility issues, they further develop an approach to deploy the interaction proxies for runtime repair and enhancement of mobile application accessibility [77]”
BackgroundCaption Crawler: Enabling Reusable Alternative Text Descriptions using Reverse Image Search
Guinness, D., Cutrell, E., & Morris, M. R. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174092 · Cited by 108
“Zhang et al. presented a proxy-based method for runtime repair of Android applications [39].”
MethodMulti-Modal Repairs of Conversational Breakdowns in Task-Oriented Dialogs
Li, T. J.-J., Chen, J., Xia, H., Mitchell, T. M., & Myers, B. A. (2020) · Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3379337.3415820 · Cited by 98
Accessible or Not? An Empirical Investigation of Android App Accessibility
Chen, S., Chen, C., Fan, L., Fan, M., Zhan, X., & Liu, Y. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3108162 · Cited by 104
“However, the associated alternative text information can be inappropriate which can confuse a user with visual impairments [85].”
BackgroundEpidemiology as a Framework for Large-Scale Mobile Application Accessibility Assessment
Ross, A. S., Zhang, X., Fogarty, J., & Wobbrock, J. O. (2017) · Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3132525.3132547 · Cited by 69
“Understanding how these factors interact and influence the accessibility of apps over time can help in improving app accessibility through development of preventative measures and post-release repairs [37].”
BackgroundSelf-citationScreen2Vec: Semantic Embedding of GUI Screens and GUI Components
Li, T. J.-J., Popowski, L., Mitchell, T. M., & Myers, B. A. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445049 · Cited by 156
“, [45]) techniques can be helpful for generating textual annotations for graphic icons so that their semantics can be represented in Screen2Vec.”
BackgroundAn Epidemiology-inspired Large-scale Analysis of Android App Accessibility
Ross, A. S., Zhang, X., Fogarty, J., & Wobbrock, J. O. (2020) · ACM Transactions on Accessible Computing · doi:10.1145/3348797 · Cited by 54
Self-citationOpen Challenges of Blind People Using Smartphones
Rodrigues, A., Nicolau, H., Montague, K., Guerreiro, J., & Guerreiro, T. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1768672 · Cited by 53
Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements
Li, Y., Li, G., He, L., Zheng, J., Li, H., & Guan, Z. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.443 · Cited by 185
Zhang, X., Ross, A. S., & Fogarty, J. A. (2018) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3242587.3242616 · Cited by 61
Self-citationMehralian, F., Salehnamadi, N., & Malek, S. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468604 · Cited by 63
Li, T. J.-J., Labutov, I., Li, X. N., Zhang, X., Shi, W., Ding, W., Mitchell, T. M., & Myers, B. A. (2018) · 2018 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2018.8506506 · Cited by 44
Self-citationMaking Mobile Augmented Reality Applications Accessible
Herskovitz, J., Wu, J., White, S., Pavel, A., Reyes, G., Guo, A., & Bigham, J. P. (2020) · Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3373625.3417006 · Cited by 46
“its expertise in accessibility evaluation [9] and fxing [68, 70] in other mediums to apply to accessible AR.”
BackgroundStateLens: A Reverse Engineering Solution for Making Existing Dynamic Touchscreens Accessible
Guo, A., Kong, J., Rivera, M., Xu, F. F., & Bigham, J. P. (2019) · Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3332165.3347873 · Cited by 35
“API. For instance, FrameWire [29] automatically extracts interaction flows from video recordings of paper prototype user tests. Using Android’s accessibility API, Sugilite [28] and Interaction Proxies [39] extract the screen structures, in order to create automation and improve mobile application accessibility. Kim et…”
MethodAccessibility in Native Mobile Applications for Users with Disabilities: A Scoping Review
Acosta-Vargas, P., Salvador-Acosta, B., Salvador-Ullauri, L., Villegas-Ch, W. E., & González, M. (2021) · Applied Sciences · doi:10.3390/app11125707 · Cited by 40
Towards Complete Icon Labeling in Mobile Applications
Chen, J., Swearngin, A., Wu, J., Barik, T., Nichols, J. W., & Zhang, X. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502073 · Cited by 51
“To address this problem, systems have been built to provide icon labels when they are not available [8, 10, 25, 36].”
BackgroundSelf-citationCodeA11y: Making AI Coding Assistants Useful for Accessible Web Development
Mowar, P., Peng, Y.-H., Wu, J., Steinfeld, A., & Bigham, J. P. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713335 · Cited by 31
“Interaction Proxies used these runtime modification strategies to “repair” Android apps by replacing inaccessible widgets with improved alternatives [92, 93].”
Zhang, X., Tran, T., Sun, Y., Culhane, I., Jain, S., Fogarty, J., & Mankoff, J. (2018) · ASSETS '18 · doi:10.1145/3234695.3236349 · Cited by 30
Self-citationA Large-Scale Longitudinal Analysis of Missing Label Accessibility Failures in Android Apps
Fok, R., Zhong, M., Ross, A. S., Fogarty, J., & Wobbrock, J. O. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502143 · Cited by 25
“Closer to our focus on missing label accessibility failures, research has demonstrated runtime repair of missing labels using interaction proxies [53] together with third-party annotation [54].”
BackgroundSelf-citationGa11y: An Automated GIF Annotation System for Visually Impaired Users
Zhang, M. R., Zhong, M., & Wobbrock, J. O. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502092 · Cited by 34
“In order to monitor on-screen contents for GIFs and insert but-tons for requesting annotations, we adopted an “interaction proxy” [46] built on the Android Accessibility API.”
BackgroundMethodSelf-citationAutomated Repair of Size-Based Inaccessibility Issues in Mobile Applications
Alotaibi, A. S., Chiou, P., & Halfond, W. G. J. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678625 · Cited by 41
“Zhang also introduced “interaction proxies” to be inserted on top of an app’s original UI for disabled users to more easily manipulate the app [24].”
BackgroundLu, T., Zheng, H., Zhang, T., Xu, X., & Guo, A. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642317 · Cited by 36
“Inspired by Interaction Proxies [80], we then demonstrate 8 Android implementations of input interventions throughout our design space, including four tap interventions (Tap Delay, Tap Prolong, Tap Shift, and Tap Double), as well as four swipe interventions (Swipe Delay, Swipe Deceleration, Swipe…”
InfluentialDetecting and localizing keyboard accessibility failures in web applications
Chiou, P. T., Alotaibi, A. S., & Halfond, W. G. J. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468581 · Cited by 22
“While the majority of this research is related to mobile accessibility failure detection, research by Zhang et al. focus on repairing these failures [61, 73, 74] as well as other deep learning [45] and search-based [38] repair.”
BackgroundAccessFixer: Enhancing GUI Accessibility for Low Vision Users With R-GCN Model
Zhang, M., Liu, H., Chen, C., Gao, G., Li, H., & Zhao, J. M. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3337421 · Cited by 20
“Afterward, as discussed in Section V.A.2, regarding the interaction proxies [47] that allow for redrawing the content of components, it could be inserted between the application’s original interface and the manifest interface that a person uses to perceive and operate the application.”
BackgroundAutomated Detection of TalkBack Interactive Accessibility Failures in Android Applications
Alotaibi, A. S., Chiou, P., & Halfond, W. G. J. (2022) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst53961.2022.00033 · Cited by 18
RePlay: Touchscreen Interaction Substitution Method for Accessible Gaming
Ahmetovic, D., Riboli, D., Bernareggi, C., & Mascetti, S. (2021) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/3447526.3472044 · Cited by 23
“In such cases, prior works propose interaction proxies [69], third-party add-ons specifcally developed for each app to repair accessibility issues, exposing previously in-accessible UI elements to the OS.”
BackgroundUnblind Your Apps: Predicting Natural-Language Labels for Mobile GUI Components by Deep LearningPreprint
Chen, J., Chen, C., Xing, Z., Xu, X., Zhu, L., Li, G., & Wang, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.00380 · Cited by 16
“For other accessibility issues, they further develop an approach to deploy the interaction proxies for runtime repair and enhancement of mobile application accessibility [77]”
Background(2019) · doi:10.1145/3308561.3353808 · Cited by 14
“Interaction Proxies [24] is a strategy to improve mobile ac cessibility with accessible screen overlays.”
BackgroundA Systematic Survey on Android API Usage for Data-driven Analytics with Smartphones
Lee, H., Park, J., & Lee, U. (2022) · ACM Computing Surveys · doi:10.1145/3530814 · Cited by 33
“[166] Interaction proxies to runtime repair and accessibility improvement of app.”
Improving Smartphone Accessibility with Personalizable Static Overlays
Rodrigues, A., Santos, A. J. D., Montague, K., & Guerreiro, T. (2017) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3132525.3132558 · Cited by 12
“Targeted application adaptations [7] can improve learnability and accessibility.”
BackgroundClick Sequence Prediction in Android Mobile Applications
Lee, S., Ha, R., & Cha, H. (2018) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2018.2868806 · Cited by 13
“[40] collected alternative texts for buttons using the accessibility service.”
MethodCummaudo, A., Barnett, S., Vasa, R., Grundy, J., & Abdelrazek, M. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3409688 · Cited by 14
“For example, addressing accessibility issues using proxy servers has been widely addressed [4, 5, 31, 34] and, more recently, they have been used to address in-browser JavaScript errors [14].”
MethodBrushLens: Hardware Interaction Proxies for Accessible Touchscreen Interface Actuation
Liang, C., Iravantchi, Y., Krolikowski, T., Geng, R., Sample, A. P., & Guo, A. (2023) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3586183.3606730 · Cited by 10
“Inspired by prior work on software interaction proxies [71] and accessibility tools for touchscreen interactions [42], our work focuses on making the interaction more accessible and risk-free through the concept of hardware interaction proxies, which introduces an intermediary layer between the user and the inaccessib…”
BackgroundMethodFully Automated HTML and Javascript Rewriting for Constructing a Self-Healing Web Proxy
Durieux, T., Hamadi, Y., & Monperrus, M. (2018) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2018.00012 · Cited by 9
Are your apps accessible? A GCN-based accessibility checker for low vision users
Zhang, M., Liu, H., Song, S., Chen, C., Huang, P., & Zhao, J. M. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107518 · Cited by 11
Promoting Self-Efficacy Through an Effective Human-Powered Nonvisual Smartphone Task Assistant
Rodrigues, A., Santos, A. J. D., Montague, K., & Guerreiro, T. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449188 · Cited by 12
“Others have proposed solutions at a systemic level that adapt how content is presented [35, 37, 47].”
BackgroundDistinguishing GUI Component States for Blind Users Using Large Language Models
Zhang, M., Liu, H., Du, C., Wang, T., Li, H., Huang, P., & Chen, C. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3722106 · Cited by 9
“These apps come from three areas: social, office, and messaging, as well as the highest rating is 4.9 and the lowest is 4.1.”
InfluentialKillough, D., Feng, J., Ching, Z. X., Wang, D., Dyava, R., Tian, Y., & Zhao, Y. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747641 · Cited by 14
Guerreiro, T., Carriço, L. M., & Rodrigues, A. (2019) · Human-computer interaction series · doi:10.1007/978-1-4471-7440-0_37 · Cited by 6
Interactive Task and Concept Learning from Natural Language Instructions and GUI DemonstrationsPreprint
Li, T. J.-J., Radensky, M., Jia, J., Singarajah, K., Mitchell, T. M., & Myers, B. A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.00031 · Cited by 7
“P UMICE uses an interaction proxy overlay [53] for recording, so that it can record all values visible on the screen, not limited to the selectable or clickable ones.”
ScaleFix: An Automated Repair of UI Scaling Accessibility Issues in Android Applications
Alotaibi, A. S., Chiou, P., Tawsif, F. M., & Halfond, W. G. J. (2023) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58846.2023.00025 · Cited by 5
“Zhang et al. created repair techniques based on using interaction proxies [43] and annotation of mobile UIs [44] to improve TalkBack accessibility.”
MethodUnderstanding mobile GUI: From pixel-words to screen-sentences
Fu, J., Zhang, X., Wang, Y., Zeng, W., & Zheng, N. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128200 · Cited by 28
Self-citationLiang, C., Liu, Y., Mott, M. E., & Guo, A. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3749484 · Cited by 10
“Their ability to introduce new functionality with minimal modifications makes them particularly valuable for tasks such as input remapping [48, 50, 92, 93] and UI automation [47].”
Baldwin, M. S., Khurana, R., McIsaac, D., Sun, Y., Tran, T., Zhang, X., Fogarty, J. A., Hayes, G. R., & Mankoff, J. C. (2019) · Human-computer interaction series · doi:10.1007/978-1-4471-7440-0_36 · Cited by 4
Self-citationHuang, T. Y., Yu, C., Shi, W., Wang, B., Yang, D. C., Zhu, Y., Li, Z., & Shi, Y. (2023) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3610929 · Cited by 6
“This method has been utilized by prior interaction proxies [61].”
BackgroundMethodInfluentialXin, T., Zhu, J., Wang, L., & Qin, X. (2023) · IEEE International Conference on Computer and Communications (ICCC) · doi:10.1109/iccc59590.2023.10507590 · Cited by 3
A Preliminary Analysis of Android Educational Game Accessibility
Martinez, J. J., Fogarty, J. A., & Froehlich, J. E. (2021) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3441852.3476532 · Cited by 2
“Although prior work has contributed to understanding [10] and repairing [13] common mobile accessibility issues, it is unclear how research in traditional apps extends to educational games.”
BackgroundSelf-citationAn epidemiology-inspired, large-scale analysis of mobile app accessibility
Ross, A. S. (2020) · ACM SIGACCESS Accessibility and Computing · doi:10.1145/3386402.3386408 · Cited by 3
“I explored the types of apps they would like to use and what accessibility barriers were most problematic [20].”
BackgroundSelf-citationViago: Exploring Visual-Audio Modality Transitions for Social Media Consumption on the Go
Chang, R.-C., Grossman, T., Rognon, C., Glueck, M., Collins, C. M., Karlson, A. K., & Surale, H. B. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747761 · Cited by 2
“Drawing from prior research [47, 154], we reverse-engineered an interaction proxy as an Android background service to simulate audio-based screen navigation.”
End user mobile task automation using multimodal programming by demonstration
Li, T. J.-J. (2017) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2017.8103491 · Cited by 1
“We plan to use an interaction proxy [11] to allow the users to talk about the UI elements on the screen during the demonstration and specify the features to use for identifying the element to operate on.”
MethodFully Automated HTML and JavaScript Rewriting for Constructing a Self‐healing Web Proxy
Durieux, T., Hamadi, Y., & Monperrus, M. (2020) · Software Testing Verification and Reliability · doi:10.1002/stvr.1731 · Cited by 1
“[37] present a technique to change the user interface of mobile applications on the fly.”
MethodRuntime Repair and Enhancement of Mobile App Accessibility
Zhang, X. (2019) · ResearchWorks at the University of Washington (University of Washington)
Self-citationLlamadas sin marcación: ayuda para discapacitados visuales y llamadas de emergencia
Hernández-Infante, J., & Gómez-Fuentes, M. D. C. (2021) · Programación matemática y software · doi:10.30973/progmat/2021.13.3/3
Caption: Generating Informative Content Labels for Image Buttons Using Next-Screen Context
Zhong, M., Mallavarapu, A., & Nie, Q. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746058.3758413 · Cited by 2
Huq, S. F., He, Z., He, Y., & Malek, S. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791293 · Cited by 5
Accessibility Issue Detection and Repair in Mobile Applications: A Systematic Literature Review
Liu, Z., Yao, D., Liu, H., Xu, C., & Huang, H. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3809496 · Cited by 2
Metadata
- Authors
- Zhang, X., Ross, A. S., Caspi, A., Fogarty, J., & Wobbrock, J. O.
- Affiliations
- University of Washington (US)
- Venue
- Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2017-05-02
- Pages
- 6024-6037
- OpenAlex
- W2610917376
- Semantic Scholar
- 8577f8716ef27982a47963c498bd12f891c70702
- Cited by
- 59
- References
- 59
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 3 · 2023: 5 · 2022: 5 · 2021: 11 · 2020: 10 · 2019: 8 · 2018: 6 · 2017: 3
- FWCI
- 14.34
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- National Science Foundation (IIS-1053868)
- Topics
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- mobile application accessibility, Android, blind and visually impaired users
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.