The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Effects of Language Modeling and its Personalization on Touchscreen Typing Performance
Open accessAbstract
Modern smartphones correct typing errors and learn user-specific words (such as proper names). Both techniques are useful, yet little has been published about their technical specifics and concrete benefits. One reason is that typing accuracy is difficult to measure empirically on a large scale. We describe a closed-loop, smart touch keyboard (STK) evaluation system that we have implemented to solve this problem. It includes a principled typing simulator for generating human-like noisy touch input, a simple-yet-effective decoder for reconstructing typed words from such spatial data, a large web-scale background language model (LM), and a method for incorporating LM personalization. Using the Enron email corpus as a personalization test set, we show for the first time at this scale that a combined spatial-language model reduces word error rate from a pre-model baseline of 38.4% down to 5.7%, and that LM personalization can improve this further to 4.6%.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Earlier work mainly focused on single word suggestions [11,12,18,34].”
From Choice Over Control: How Users Write with Large Language Models using Diegetic and Non-Diegetic Prompting (2023) · cites this paper as [11]
In the citing paper
“Traditionally, systems mainly use diegetic information, that is, they predict text based (only) on the preceding text [11,12,18,34].”
From Choice Over Control: How Users Write with Large Language Models using Diegetic and Non-Diegetic Prompting (2023) · cites this paper as [11]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Estimation of probabilities from sparse data for the language model component of a speech recognizer
Katz, S. M. (1987) · IEEE Transactions on Acoustics Speech and Signal Processing · doi:10.1109/tassp.1987.1165125 · Cited by 1,664
Improved backing-off for M-gram language modeling
Kneser, R., & Ney, H. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1995.479394 · Cited by 1,471
A bit of progress in language modeling
Goodman, J. (2001) · Computer Speech and Language · doi:10.1006/csla.2001.0174 · Cited by 744
Klimt, B., & Yang, Y. (2004) · Conference on Email and Anti-Spam · Cited by 575
Text Entry for Mobile Computing: Models and Methods, Theory and Practice
MacKenzie, I. S., & Soukoreff, R. W. (2002) · Human-Computer Interaction · doi:10.1207/s15327051hci172&3_2 · Cited by 571
A cache-based natural language model for speech recognition
Kühn, R., & Mori, R. D. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.56193 · Cited by 502
Kristensson, P.-O., & Zhai, S. (2004) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/1029632.1029640 · Cited by 419
Self-citationDasher---a data entry interface using continuous gestures and language models
Ward, D. J., Blackwell, A. F., & Mackay, D. (2000) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/354401.354427 · Cited by 416
Shorthand writing on stylus keyboard
Zhai, S., & Kristensson, P.-O. (2003) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642630 · Cited by 371
Self-citationStatistical language model adaptation: review and perspectives
Bellegarda, J. R. (2003) · Speech Communication · doi:10.1016/j.specom.2003.08.002 · Cited by 305
Holz, C., & Baudisch, P. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979308 · Cited by 278
Touch behavior with different postures on soft smartphone keyboards
Azenkot, S., & Zhai, S. (2012) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/2371574.2371612 · Cited by 225
Self-citationA versatile dataset for text entry evaluations based on genuine mobile emails
Vertanen, K., & Kristensson, P. O. (2011) · Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services · doi:10.1145/2037373.2037418 · Cited by 177
Language model adaptation using mixtures and an exponentially decaying cache
Clarkson, P. R., & Robinson, A. J. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1997.596049 · Cited by 171
Zhai, S., & Kristensson, P. O. (2012) · Communications of the ACM · doi:10.1145/2330667.2330689 · Cited by 169
Self-citationLanguage modeling for soft keyboards
Goodman, J., Venolia, G. D., Steury, K. R., & Parker, C. R. (2002) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/502716.502753 · Cited by 162
Evaluation Metrics For Language Models
Chen, S. F., Beeferman, D., & Rosenfeld, R. (2018) · doi:10.1184/r1/6605324.v1 · Cited by 148
Evaluation Metrics For Language Models
Chen, S. F., Beeferman, D., & Rosenfeld, R. (2018) · doi:10.1184/r1/6605324 · Cited by 145
Domain Adaptation of Rule-Based Annotators for Named-Entity Recognition Tasks
Chiticariu, L., Krishnamurthy, R., Li, Y., Reiss, F. R., & Vaithyanathan, S. (2010) · Cited by 129
Typing on flat glass: examining ten-finger expert typing patterns on touch surfaces
Findlater, L., Wobbrock, J. O., & Wigdor, D. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979301 · Cited by 115
Performance comparisons of phrase sets and presentation styles for text entry evaluations
Kristensson, P. O., & Vertanen, K. (2012) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/2166966.2166972 · Cited by 107
Zajic, D. M., Dorr, B. J., & Lin, J. (2007) · Information Processing & Management · doi:10.1016/j.ipm.2007.09.007 · Cited by 93
Uncertain text entry on mobile devices
Weir, D., Pohl, H., Rogers, S., Vertanen, K., & Kristensson, P. O. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557412 · Cited by 90
Bi, X., Ouyang, T. Y., & Zhai, S. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557414 · Cited by 61
Self-citationMaking touchscreen keyboards adaptive to keys, hand postures, and individuals
Yin, Y., Ouyang, T. Y., Partridge, K., & Zhai, S. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2481384 · Cited by 56
Self-citationStudy on interaction between entropy pruning and kneser-ney smoothing
Chelba, C. I., Brants, T., Neveitt, W., & Xu, P. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-525 · Cited by 39
Self-citationDeMori, R., & Federico, M. (1999) · Springer eBooks · doi:10.1007/978-3-642-60087-6_26 · Cited by 37
Sampling representative phrase sets for text entry experiments
Paek, T., & Hsu, B.-J. P. (2011) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979304 · Cited by 30
Word-based predictive text entry using adaptive language models
TANAKA-ISHII, K. (2007) · Natural Language Engineering · doi:10.1017/s1351324905004080 · Cited by 29
Towards better NLP system evaluation
Jones, K. S. (1994) · doi:10.3115/1075812.1075833 · Cited by 28
Asynchronous multimodal text entry using speech and gesture keyboards
Kristensson, P. O., & Vertanen, K. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-236 · Cited by 27
Alphabetic Data Entry via the Touch-Tone Pad: A Comment
Smith, S. L., & Goodwin, N. C. (1971) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/001872087101300212 · Cited by 23
Domain Adaptation to Summarize Human Conversations
Sandu, O. A., Carenini, G., Murray, G., & Ng, R. T. (2010) · Cited by 17
Wordn-grams for cluster keyboards
Klarlund, N., & Riley, M. (2003) · doi:10.3115/1628195.1628202 · Cited by 16
Improving word-recognizers using an interactive lexicon with active and passive words
Kristensson, P. O., & Zhai, S. (2008) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/1378773.1378828 · Cited by 6
Self-citation
Research works printed but not linked: 9
- 4 . Charniak E. Blaheta D. Ge N. Hall K. Hale J. and Johnson M. BLLIP 1987--89 Wall Street Journal Corpus Release 1. 4. Charniak E. Blaheta D. Ge N. Hall K. Hale J. and Johnson M. BLLIP 1987--89 Wall Street Journal Corpus Release 1.
- Chelba C., 2010, Proc. Interspeech, ISCA
- 10.5555/839274.839354 · DOI
- Jurafsky D., 2008, Speech & Language Processing
- 10.5555/1628195.1628202 · DOI
- Klimt B., 2004, Proceedings of the Conference on Email and Anti-Spam, CEAS '04
- Kristensson P. O., 2011, Proc. Interspeech
- Sandu O., 2010, Proceedings of the Workshop on Domain Adaptation for Natural Language Processing, Association for Computational Linguistics
- Stolcke A., 1998, Proceedings of News Transcription and Understanding Workshop, DARPA
Cited by
Sorted by citations
Co-Writing with Opinionated Language Models Affects Users’ Views
Jakesch, M., Bhat, A., Buschek, D., Zalmanson, L., & Naaman, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581196 · Cited by 265
“They suggest one to three subsequent words based on underlying likelihood distributions [40, 43, 49, 89].”
BackgroundDraxler, F., Werner, A., Lehmann, F., Hoppe, M., Schmidt, A., Buschek, D., & Welsch, R. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3637875 · Cited by 186
“Concretely, this included suggesting whole words, which users can then select instead of typing them letter by letter [35, 51].”
BackgroundA Design Space for Intelligent and Interactive Writing Assistants
Lee, M., Gero, K. I., Chung, J. J. Y., Shum, S. B., Raheja, V., Shen, H., Venugopalan, S., Wambsganss, T., Zhou, D., Alghamdi, E. A., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642697 · Cited by 136
“Additionally, uses of writing assistants have begun to affect labor markets [24, 69, 182, 183], signifying substantial societal consequences and evidencing the need for monitoring such effects.”
Where to Hide a Stolen Elephant: Leaps in Creative Writing with Multimodal Machine Intelligence
Singh, N., Bernal, G., Savchenko, D., & Glassman, E. L. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3511599 · Cited by 119
LaMP: When Large Language Models Meet Personalization
Salemi, A., Mysore, S., Bendersky, M., & Zamani, H. (2023) · arXiv · doi:10.48550/arxiv.2304.11406 · Cited by 109
Metadata
- Authors
- Fowler, A., Partridge, K., Chelba, C., Bi, X., Ouyang, T., & Zhai, S.
- Affiliations
- Oregon Health & Science University (US)
- Google (United States) (US)
- Venue
- Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2015-04-18
- Pages
- 649-658
- OpenAlex
- W2051780479
- Cited by
- 127
- References
- 35
- Citations by year
- 2026: 2 · 2025: 8 · 2024: 54 · 2023: 10 · 2022: 10 · 2021: 10 · 2020: 6 · 2019: 8 · 2018: 6 · 2017: 7 · 2016: 6
- FWCI
- 4.79
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- National Institutes of Health (R01 DC009834)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Keywords
- language modeling, personalization, word error rate, typing accuracy, proper names
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
