The vocal joystick: evaluation of voice-based cursor control techniques
The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Mouse control has become a crucial aspect of many modern day computer interactions. This poses a challenge for individuals with motor impairments or those whose use of hands are restricted due to situational constraints. We present a system called the Vocal Joystick which allows the user to continuously control the mouse cursor by varying vocal parameters such as vowel quality, loudness and pitch. A survey of existing cursor control methods is presented to highlight the key characteristics of the Vocal Joystick. Evaluations were conducted to characterize expert performance capability of the Vocal Joystick, and to compare novice user performance and preference for the Vocal Joystick and two other existing speech based cursor control methods. Our results show that Fitts' law is a good predictor of the speedaccuracy tradeoff for the Vocal Joystick, and suggests that the optimal performance of the Vocal Joystick may be comparable to that of a conventional hand-operated joystick. Novice user evaluations show that the Vocal Joystick can be used by people without extensive training, and that it presents a viable alternative to existing speech-based cursor control methods.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The use of non-verbal voice input for interactive control has been widely explored [52], [53].”
From Hands-Free Accessible Digital Musical Instruments: Conceptual Framework, Challenges, and Perspectives (2020) · cites this paper as [53]
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 36,808
The information capacity of the human motor system in controlling the amplitude of movement.
Fitts, P. M. (1954) · Journal of Experimental Psychology · doi:10.1037/h0055392 · Cited by 6,466 · cited 1×
“Fitts’ law [7] provides a mathematical formulation, based on Shannon’s fundamental theorem of communication systems [19].”
The information capacity of the human motor system in controlling the amplitude of movement.
Fitts, P. M. (1992) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.121.3.262 · Cited by 6,278
Fitts' Law as a Research and Design Tool in Human-Computer Interaction
MacKenzie, I. S. (1992) · Human–Computer Interaction · doi:10.1207/s15327051hci0701_3 · Cited by 1,637 · cited 4×
“For our study, we chose to use a minor variation of the formulation as proposed by MacKenzie [12] that has been shown to be slightly more accurate and theoretically sound [3], where ID is defined as:”
MethodInfluentialCARD, S. K., ENGLISH, W. K., & BURR, B. J. (1978) · Ergonomics · doi:10.1080/00140137808931762 · Cited by 887
What you look at is what you get: eye movement-based interaction techniques
Jacob, R. J. K. (1990) · Proceedings of the SIGCHI conference on Human factors in computing systems Empowering people - CHI '90 · doi:10.1145/97243.97246 · Cited by 712 · cited 1×
“Although these devices provide an attractive alternative for individuals with very limited mobility, it requires specialized hardware to be attached to the computer, and often involve extra steps in the interaction process to work around the “Midas touch” problem [10].”
Voice as sound: using non-verbal voice input for interactive control
Igarashi, T., & Hughes, J. F. (2001) · Proceedings of the 14th annual ACM symposium on User interface software and technology · doi:10.1145/502348.502372 · Cited by 154 · cited 1×
“proposed several techniques for using nonverbal voice features such as utterance duration, pitch, and discrete sound frequency to control the rate of change of some interface elements such as when scrolling a map [9].”
MethodComparison of Six Cursor Control Devices Based on Fitts' Law Models
Epps, B. W. (1986) · Proceedings of the Human Factors Society Annual Meeting · doi:10.1177/154193128603000403 · Cited by 147 · cited 1×
“Joystick [6] (displacement; velocity control) 0.”
A Method for Evaluating Head-Controlled Computer Input Devices Using Fitts' Law
Radwin, R. G., Vanderheiden, G. C., & Lin, M.-L. (1990) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/001872089003200405 · Cited by 125 · cited 1×
“Ultrasonic head pointer [17] 0.”
Research works printed but not linked: 4
- 10.5555/129837 · doi:10.5555/129837
- C. de Mauro , M. Gori , M. Maggini , and E. Martinelli . Easy access to graphical interfaces by voice mouse. Technical report , Universit`a di Siena , 2001 . Available from the author at: maggini@dii.unisi.it. C. de Mauro, M. Gori, M. Maggini, and E. Martinelli. Easy access to graphical interfaces by voice mouse. Technical report, Universit`a di Siena, 2001. Available from the author at: maggini@dii.unisi.it.
- 10.5555/646814.708131 · doi:10.5555/646814.708131
- C. E. Shannon and W. Weaver . The mathematical theory of communicaton . Urbana University of Illinois Press , Urbana, Illinois , 1949 . C. E. Shannon and W. Weaver. The mathematical theory of communicaton. Urbana University of Illinois Press, Urbana, Illinois, 1949.
Cited by
Ability-Based Design: Concept, Principles and Examples
Wobbrock, J. O., Kane, S. K., Gajos, K. Z., Harada, S., & Froehlich, J. (2011) · ACM Transactions on Accessible Computing · doi:10.1145/1952383.1952384 · Cited by 567
“A vowel-sound mapping was used by the underlying Vocal Joystick [Harada et al. 2006], providing a vowel map (Figure 7) in which each sound could be morphed into the sounds around it, making it possible for a user to smoothly move the paintbrush in any direction.”
MethodSelf-citationYuan, B., Folmer, E., & Harris, F. C. (2011) · Universal Access in the Information Society · doi:10.1007/s10209-010-0189-5 · Cited by 372
“Software solutions exist, such as a vocal joystick [40,38], that enables the player to control a cursor”
BackgroundAbracadabra: wireless, high-precision, and unpowered finger input for very small mobile devices
Harrison, C., & Hudson, S. E. (2009) · Proceedings of the 22nd annual ACM symposium on User interface software and technology · doi:10.1145/1622176.1622199 · Cited by 268
“Other potential approaches have employed cameras [10], microphones [5], proximity sensors [2], magnetic tags [7], and accelerometers [3].”
BackgroundAutomatically generating user interfaces adapted to users' motor and vision capabilities
Gajos, K. Z., Wobbrock, J. O., & Weld, D. S. (2007) · Proceedings of the 20th annual ACM symposium on User interface software and technology · doi:10.1145/1294211.1294253 · Cited by 240
“VJ01 Vocal Joystick [14]”
Review of Three-Dimensional Human-Computer Interaction with Focus on the Leap Motion Controller
Bachmann, D., Weichert, F., & Rinkenauer, G. (2018) · Sensors · doi:10.3390/s18072194 · Cited by 167
“[34] suggested a vocal joystick (non-speech voice input) as a continuous input device.”
Background
Metadata
- Authors
- Harada, S., Landay, J. A., Malkin, J., Li, X., & Bilmes, J. A.
- Affiliations
- University of Washington (US)
- Venue
- Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility
- Publisher
- ACM
- Published
- 2006-10-23
- Pages
- 197-204
- OpenAlex
- W1983020750
- Semantic Scholar
- 49898052072faf663b1413675c95e1b5a0bd5430
- Cited by
- 95
- References
- 20
- Citations by year
- 2025: 1 · 2024: 2 · 2023: 3 · 2022: 5 · 2020: 5 · 2019: 3 · 2018: 7 · 2017: 4 · 2016: 4 · 2015: 4 · 2014: 3 · 2013: 7 · 2012: 7
- FWCI
- 5.52
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- mouse control, motor impairment, assistive technology, vowel quality, pitch, loudness, Fitts' law, speed-accuracy trade-off, user preferences
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The vocal joystick: evaluation of voice-based cursor control techniques. https://app.cochat.ai/papers/10.1145/1168987.1169021
- The paper
- Harada, S., Landay, J. A., Malkin, J., Li, X., & Bilmes, J. A. (2006). The vocal joystick: evaluation of voice-based cursor control techniques. Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility. https://doi.org/10.1145/1168987.1169021
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.