The DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Word Clarity as a Metric in Sampling Keyboard Test Sets
Abstract
Test sets play an essential role in evaluating text entry techniques. In this paper, we argue that in addition to the widely adopted metric of bigram representativeness and memorability, word clarity should also be considered as a metric when creating test sets from the target dataset. Word clarity quantifies the extent to which a word is likely to confuse with other words on a keyboard. We formally define word clarity, derive equations calculating it, and both theoretically and empirically show that word clarity has a significant effect on text entry performance: it can yield up to 26.4% difference in error rate, and 25% difference in input speed. We later propose a Pareto optimization method for sampling test sets with different sizes, which optimizes the word clarity and bigram representativeness, and memorability of the test set. The obtained test sets are published on the Internet.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In one evaluation study, the dataset proposed in [66] was used. One evaluation study selected words from the Open American National Corpus (ANC), and one evaluation study used Pinyin, a phonetic spelling system in Roman characters for inputting Chinese characters. Finally, one evaluation study did not provide details about the used phrase set.”
From A Review of Design and Evaluation Practices in Mobile Text Entry for Visually Impaired and Blind Persons (2023) · cites this paper as [66]
Citations
Cites
The Kolmogorov-Smirnov Test for Goodness of Fit
Massey, F. J. (1951) · Journal of the American Statistical Association · doi:10.1080/01621459.1951.10500769 · Cited by 7,210
The information capacity of the human motor system in controlling the amplitude of movement.
Fitts, P. M. (1954) · Journal of Experimental Psychology · doi:10.1037/h0055392 · Cited by 6,466 · cited 1×
“Researchers usually calculate the movement efficiency (MT ) as the sum of Fitts’ law [12] movement time (Ti j) between all pairs of letters (bigrams) weighted by bigram probabilities Pi j calculated from a language corpus:”
BackgroundThe information capacity of the human motor system in controlling the amplitude of movement.
Fitts, P. M. (1992) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.121.3.262 · Cited by 6,278
The Kolmogorov-Smirnov Test for Goodness of Fit
Massey, F. J. (1951) · Journal of the American Statistical Association · doi:10.2307/2280095 · Cited by 3,690 · cited 1×
“clarity of Ŝ and that of U based on the Kolmogorov-Smirnov test (K-S test) [28].”
MethodPhrase sets for evaluating text entry techniques
MacKenzie, I. S., & Soukoreff, R. W. (2003) · CHI '03 extended abstracts on Human factors in computing systems - CHI '03 · doi:10.1145/765891.765971 · Cited by 825 · cited 10×
“Optimization Metrics When sampling phrase sets for text entry experiments, memorability and representativeness are two metrics that have been widely adopted by researchers [23, 25, 33, 39].”
BackgroundMethodInfluentialKlimt, B., & Yang, Y. (2004) · Conference on Email and Anti-Spam · Cited by 575
Metrics for text entry research: an evaluation of MSD and KSPC, and a new unified error metric
Soukoreff, R. W., & MacKenzie, I. S. (2003) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642632 · Cited by 398 · cited 2×
“Consistent with existing works, subjects tended to fix most of the errors and left few in the final submitted string [24, 38, 42].”
MethodResultFat Finger Worries: How Older and Younger Users Physically Interact with PDAs
Siek, K. A., Rogers, Y., & Connelly, K. H. (2005) · Lecture Notes in Computer Science · doi:10.1007/11555261_24 · Cited by 337 · cited 1×
“INTRODUCTION Due to the “fat-finger problem” [35] and lack of tactile feedback, text entry has long been a challenge for touchscreen devices [2].”
BackgroundPredicting text entry speed on mobile phones
Silfverberg, M., MacKenzie, I. S., & Korhonen, P. (2000) · Proceedings of the SIGCHI conference on Human Factors in Computing Systems · doi:10.1145/332040.332044 · Cited by 305 · cited 1×
“Meanwhile, bigram probability has been extensively used for both predicting the text rate [8, 10, 26, 36] and for designing representative phrase sets [31, 39].”
MethodPatterns of entry and correction in large vocabulary continuous speech recognition systems
Karat, C.-M., Halverson, C., Horn, D., & Karat, J. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303160 · Cited by 275 · cited 1×
“western novel [18], Linux operating system [17], and news [43]).”
Background
Research works printed but not linked: 1
- Paul Nation and Robert Waring . 1997. Vocabulary size, text coverage and word lists. Vocabulary: Description, acquisition and pedagogy 14 ( 1997 ), 6--19. Paul Nation and Robert Waring. 1997. Vocabulary size, text coverage and word lists. Vocabulary: Description, acquisition and pedagogy 14 (1997), 6--19.
Cited by
Observations on Typing from 136 Million Keystrokes
Dhakal, V., Feit, A. M., Kristensson, P. O., & Oulasvirta, A. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174220 · Cited by 226
“[42] proposed word clarity as a metric for sampling test sets for touch screen keyboards.”
BackgroundWrisText: One-handed Text Entry on Smartwatch using Wrist Gestures
Gong, J., Xu, Z., Guo, Q., Seyed, T., Chen, X. '., Bi, X., & Yang, X.-D. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173755 · Cited by 101
“In each session, participants typed 18 phrases with each hand posture, the same across all participants, randomly picked from [60].”
Self-citationBiTipText: Bimanual Eyes-Free Text Entry on a Fingertip Keyboard
Xu, Z., Chen, W., Zhao, D., Luo, J., Wu, T.-Y., Gong, J., Yin, S., Zhai, J., & Yang, X.-D. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376306 · Cited by 52
“We took a simplified approach similar to the one used in previous work for word clarity on a soft keyboard [39], where 𝑥 and 𝑦 coordinates were assumed to be independent to each other, 𝑠 (cid:3036) and the center of the touch point distribution were also assumed to be in the center of the target…”
MethodVelociWatch: Designing and Evaluating a Virtual Keyboard for the Input of Challenging Text
Vertanen, K. D., Gaines, D. C., Fletcher, C., Stanage, A. M., Watling, R., & Kristensson, P. O. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300821 · Cited by 46
“Yi et al. [32] proposed a word clarity metric based on geometric pattern matching [16] for sampling phrases containing words that might be confused due to their geometric proximity.”
BackgroundThe Impact of Word, Multiple Word, and Sentence Input on Virtual Keyboard Decoding Performance
Vertanen, K. D., Fletcher, C., Gaines, D. C., Gould, J., & Kristensson, P. O. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174200 · Cited by 43
“Later research has investigated further ways of sampling representative phrases based on criteria such as word clarity [36].”
Background
Metadata
- Authors
- Yi, X., Yu, C., Shi, W., Bi, X., & Shi, Y.
- Affiliations
- Tsinghua University (CN)
- Stony Brook University (US)
- Venue
- Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2017-05-02
- Pages
- 4216-4228
- OpenAlex
- W2611531483
- Cited by
- 26
- References
- 40
- Citations by year
- 2025: 3 · 2024: 1 · 2023: 4 · 2022: 1 · 2021: 3 · 2020: 3 · 2019: 4 · 2018: 7
- FWCI
- 2.89
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- Natural Science Foundation of China (61272230,61572276)
- National Key Research and Development Plan (2016YFB1001200)
- Tsinghua University Research Funding (20151080408)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Information Systems › User Authentication and Security Systems
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Keywords
- text entry, memorability, input speed, multi-objective optimization, word confusion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Word Clarity as a Metric in Sampling Keyboard Test Sets. https://app.cochat.ai/papers/10.1145/3025453.3025701
- The paper
- Yi, X., Yu, C., Shi, W., Bi, X., & Shi, Y. (2017). Word Clarity as a Metric in Sampling Keyboard Test Sets. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3025453.3025701
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.