Metrics for text entry research: an evaluation of MSD and KSPC, and a new unified error metric
Abstract
We describe and identify shortcomings in two statistics recently introduced to measure accuracy in text entry evaluations: the minimum string distance (MSD) error rate and keystrokes per character (KSPC). To overcome the weaknesses, a new framework for error analysis is developed and demonstrated. It combines the analysis of the presented text, input stream (keystrokes), and transcribed text. New statistics include a unified total error rate, combining two constituent error rates: the corrected error rate (errors committed but corrected) and the not corrected error rate (errors left in the transcribed text). The framework includes other measures including error correction efficiency, participant conscientiousness, utilised bandwidth, and wasted bandwidth. A text entry study demonstrating the new methodology is described.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior works on non-visual text-entry focus on overall input performance metrics such as WPM and MSD error rates to compare input methods.
“What is common amongst these works is that they focus on the overall input performance metrics such as words per minute (WPM) and minimum string distance (MSD) error rates [21] to compare input methods.”
From Typing Performance of Blind Users (2015) · cites this paper as [21]
Cited by
Sorted by citations
Phrase sets for evaluating text entry techniques
MacKenzie, I. S., & Soukoreff, R. W. (2003) · CHI '03 extended abstracts on Human factors in computing systems - CHI '03 · doi:10.1145/765891.765971 · Cited by 825 · see citing works
Self-citationAnalysis of text entry performance metrics
Arif, A. S., & Stuerzlinger, W. (2009) · 2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH) · doi:10.1109/tic-sth.2009.5444533 · Cited by 206 · see citing works
Analyzing the input stream for character- level errors in unconstrained text entry evaluations
Wobbrock, J. O., & Myers, B. A. (2006) · ACM Transactions on Computer-Human Interaction · doi:10.1145/1188816.1188819 · Cited by 200 · see citing works
WatchWriter: Tap and Gesture Typing on a Smartwatch Miniature Keyboard with Statistical Decoding
Gordon, M., Ouyang, T., & Zhai, S. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858242 · Cited by 150 · see citing works
An evaluation of BrailleTouch: mobile touchscreen text entry for the visually impaired
Southern, C., Clawson, J., Frey, B., Abowd, G., & Romero, M. (2012) · Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services · doi:10.1145/2371574.2371623 · Cited by 125 · see citing works
Personalized input: improving ten-finger touchscreen typing through automatic adaptation
Findlater, L., & Wobbrock, J. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208520 · Cited by 106 · see citing works
Complementing Text Entry Evaluations with a Composition Task
Vertanen, K., & Kristensson, P. O. (2014) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2555691 · Cited by 62 · see citing works
An exploratory study of eye typing fundamentals: dwell time, text entry rate, errors, and workload
Räihä, K.-J., & Ovaska, S. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208711 · Cited by 60 · see citing works
Belatar, M., & Poirier, F. (2008) · Proceedings of the 10th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/1414471.1414510 · Cited by 40 · see citing works
Typing Performance of Blind Users
Nicolau, H., Montague, K., Guerreiro, T., Rodrigues, A., & Hanson, V. L. (2015) · Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility - ASSETS '15 · doi:10.1145/2700648.2809861 · Cited by 37 · see citing works
Alnfiai, M., & Sampalli, S. (2016) · Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/2982142.2982161 · Cited by 33 · see citing works
Word Clarity as a Metric in Sampling Keyboard Test Sets
Yi, X., Yu, C., Shi, W., Bi, X., & Shi, Y. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025701 · Cited by 26 · see citing works
Alnfiai, M., & Sampali, S. (2017) · 2017 International Conference on Computer and Applications (ICCA) · doi:10.1109/comapp.2017.8079740 · Cited by 20 · see citing works
Hybrid-Brailler: Combining Physical and Gestural Interaction for Mobile Braille Input and Editing
Trindade, D., Rodrigues, A., Guerreiro, T., & Nicolau, H. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173601 · Cited by 19 · see citing works
Senorita: A Chorded Keyboard for Sighted, Low Vision, and Blind Mobile Users
Rakhmetulla, G., & Arif, A. S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376576 · Cited by 14 · see citing works
- Authors
- Soukoreff, R. W., & MacKenzie, I. S.
- Affiliations
- York University (CA)
- ORCID
- MacKenzie, I. Scott
- Venue
- Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2003-04-05
- Pages
- 113-120
- OpenAlex
- W2103751734
- Cited by
- 398
- References
- 0
- Citations by year
- 2026: 11 · 2025: 17 · 2024: 20 · 2023: 13 · 2022: 21 · 2021: 17 · 2020: 19 · 2019: 25 · 2018: 21 · 2017: 22 · 2016: 17 · 2015: 22 · 2014: 15 · 2013: 21 · 2012: 21
- FWCI
- 10.56
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- Natural Sciences and Engineering Research Council of Canada
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Information Systems › User Authentication and Security Systems
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Keywords
- text entry
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
