Efficacy of personalized models in discriminating high cognitive demand conditions using text-based interactions
Open accessAbstract
Although high cognitive demand conditions can impair psychological, physical, and behavioral processes without appropriate management, current measurement methods are too cumbersome for continuous monitoring of cognitive demand, and do not account for individual differences. This research uses keystroke and linguistic markers of typed text to construct individualized models of cognitive demand response to discriminate high and low cognitive demand conditions, the results of which can have implications for design of cognitive demand monitoring systems for personalized health management. We constructed within-subject models of cognitive demand response for nine participants and one between-subjects model based on 20 participants. The AUCs for personalized models ranged from 0.679 to 0.953 (Mean=0.826, SD=0.085), significantly higher than chance (p < 0.0001) and the 0.714 AUC for the generic model (p=0.002). Although the features in each model were different, the most common features across models are rate of negative emotion, lexical diversity, rate of words over six letters, and word count. These results confirm significant individual differences in cognitive demand response and suggest that those developing measurement methods used in a monitoring system should consider adaptation to individual characteristics. Our research operationalizes the effects of cognitive demand on HCI and contributes a unique combination of text and keystroke features used to detect high cognitive demand situations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(1990) · Choice Reviews Online · doi:10.5860/choice.27-3338 · Cited by 35,305 · see citing works
Data Mining: Practical Machine Learning Tools and Techniques
Witten, I. H., Frank, E., & Hall, M. A. (2011) · Elsevier eBooks · doi:10.1016/c2009-0-19715-5 · Cited by 25,493 · see citing works
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003 · see citing works
Hosmer, D. W., Lemeshow, S., & Sturdivant, R. X. (2013) · Wiley Series in Probability and Statistics · doi:10.1002/9781118548387 · Cited by 13,861 · see citing works
Cognitive Load During Problem Solving: Effects on Learning
Sweller, J. (1988) · Cognitive Science · doi:10.1016/0364-0213(88)90023-7 · Cited by 11,048 · cited 2× · see citing works
“2007), Cognitive load is the perception of the total cognitive effort expended on a task (Sweller 1988).”
BackgroundWitten, I. H. (2008) · Cited by 9,115 · see citing works
Statistical pattern recognition: a review
Jain, A. K., Duin, P., & Mao, J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.824819 · Cited by 6,795 · see citing works
Anxiety and cognitive performance: Attentional control theory
Eysenck, M. W., Derakshan, N., Santos, R., & Calvo, M. G. (2007) · Emotion · doi:10.1037/1528-3542.7.2.336 · Cited by 5,289 · see citing works
Karasek, R. A., Brisson, C., Kawakami, N., Houtman, I. L. D., Bongers, P. M., & Amick, B. C. (1998) · Journal of Occupational Health Psychology · doi:10.1037//1076-8998.3.4.322 · Cited by 4,022 · see citing works
Applied Logistic Regression Analysis
Ziegel, E. R., & Menard, S. (1996) · Technometrics · doi:10.1080/00401706.1996.10484485 · Cited by 3,923 · see citing works
N-back working memory paradigm: a meta-analysis of normative functional neuroimaging studies.
Owen, A. M., McMillan, K. M., Laird, A. R., & Bullmore, E. (2005) · Human Brain Mapping · doi:10.1002/hbm.20131 · Cited by 3,560 · see citing works
Natural language processing with Python
Bird, S., Klein, E., & Loper, E. (2009) · CERN Document Server (European Organization for Nuclear Research) · Cited by 3,472 · see citing works
Karasek, R. A., Brisson, C., Kawakami, N., Houtman, I. L. D., Bongers, P. M., & Amick, B. C. (1998) · Journal of Occupational Health Psychology · doi:10.1037/1076-8998.3.4.322 · Cited by 3,137 · see citing works
Applied Logistic Regression Analysis
Menard, S. (2002) · Sage eBooks · doi:10.4135/9781412983433 · Cited by 3,011 · see citing works
DePaulo, B. M., Lindsay, J. J., Malone, B. E., Muhlenbruck, L., Charlton, K., & Cooper, H. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.1.74 · Cited by 2,479 · see citing works
Working memory and language: an overview
Baddeley, A. (2003) · Journal of Communication Disorders · doi:10.1016/s0021-9924(03)00019-4 · Cited by 2,346 · see citing works
Linguistic Inquiry and Word Count (LIWC)
Chung, C. K., & Pennebaker, J. W. (2012) · IGI Global eBooks · doi:10.4018/978-1-60960-741-8.ch012 · Cited by 2,152 · see citing works
Lupien, S. J., Maheu, F., Tu, M., Fiocco, A., & Schramek, T. E. (2007) · Brain and Cognition · doi:10.1016/j.bandc.2007.02.007 · Cited by 1,474 · see citing works
Verbal and Nonverbal Communication of Deception
Zuckerman, M., DePaulo, B. M., & Rosenthal, R. (1981) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60369-x · Cited by 1,341 · see citing works
Smith, A. (1973) · PsycTESTS Dataset · doi:10.1037/t27513-000 · Cited by 1,314 · see citing works
A Dynamic Model of Stress and Sustained Attention
Hancock, P. A. (1989) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/001872088903100503 · Cited by 874 · see citing works
Strategies for Measuring Stress in Studies of Psychiatric and Physical Disorders
Cohen, S., Kessler, R. C., & Gordon, L. U. (1995) · doi:10.1093/oso/9780195086416.003.0001 · Cited by 782 · see citing works
Keystroke dynamics as a biometric for authentication
Monrose, F., & Rubin, A. D. (2000) · Future Generation Computer Systems · doi:10.1016/s0167-739x(99)00059-x · Cited by 748 · see citing works
Matthews, G., Campbell, S. E., Falconer, S., Joyner, L. A., Huggins, J., Gilliland, K., Grier, R. A., & Warm, J. S. (2002) · Emotion · doi:10.1037/1528-3542.2.4.315 · Cited by 596 · see citing works
Lu, H., Frauendorfer, D., Rabbi, M., Mast, M. S., Chittaranjan, G. T., Campbell, A. T., Gática-Pérez, D., & Choudhury, T. (2012) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2370216.2370270 · Cited by 511 · see citing works
Keystroke analysis of free text
Gunetti, D., & Picardi, C. (2005) · ACM Transactions on Information and System Security · doi:10.1145/1085126.1085129 · Cited by 476 · see citing works
Adult age differences in working memory
Dobbs, A. R., & Rule, B. G. (1989) · Psychology and Aging · doi:10.1037/0882-7974.4.4.500 · Cited by 466 · see citing works
Zhou, L., Burgoon, J. K., Nunamaker, J. F., & Twitchell, D. (2004) · Group Decision and Negotiation · doi:10.1023/b:grup.0000011944.62889.6f · Cited by 440 · see citing works
The Relationship Between Workplace Stressors and Mortality and Health Costs in the United States
Goh, J., Pfeffer, J., & Zenios, S. A. (2015) · Management Science · doi:10.1287/mnsc.2014.2115 · Cited by 408 · see citing works
Adult age differences in working memory.
Dobbs, A. R., & Rule, B. G. (1989) · Psychology and Aging · doi:10.1037//0882-7974.4.4.500 · Cited by 389 · see citing works
Identifying emotional states using keystroke dynamics
Epp, C., Lippold, M., & Mandryk, R. L. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979046 · Cited by 364 · see citing works
Towards a tool for the Subjective Assessment of Speech System Interfaces (SASSI)
Hone, K., & Graham, R. (2000) · Natural Language Engineering · doi:10.1017/s1351324900002497 · Cited by 355 · see citing works
Traditional and raw task load index (TLX) correlations: Are paired comparisons necessary? In A
Byers, J. C., Bittner, A. C., & Hill, S. G. (1989) · Medical Entomology and Zoology · Cited by 326 · see citing works
Human-centered design meets cognitive load theory
Oviatt, S. L. (2006) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1180639.1180831 · Cited by 322 · see citing works
Mehler, B. L., Reimer, B., & Coughlin, J. F. (2012) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720812442086 · Cited by 298 · see citing works
Measuring occupational stress: Development of the Pressure Management Indicator.
Williams, S., & Cooper, C. L. (1998) · Journal of Occupational Health Psychology · doi:10.1037//1076-8998.3.4.306 · Cited by 277 · see citing works
A Comparison of Classification Methods for Predicting Deception in Computer-Mediated Communication
Zhou, L., Burgoon, J. K., Twitchell, D. P., Qin, T., & Nunamaker, J. F. (2004) · Journal of Management Information Systems · doi:10.1080/07421222.2004.11045779 · Cited by 249 · see citing works
SmartCar: detecting driver stress
Healey, J. H., & Picard, R. W. (2002) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2000.902898 · Cited by 247 · see citing works
Stress management techniques: evidence-based procedures that reduce stress and promote health
Varvogli, L., & Darviri, C. (2011) · Cited by 242 · see citing works
Automated stress detection using keystroke and linguistic features: An exploratory study
Vizer, L. M., Zhou, L., & Sears, A. (2009) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2009.07.005 · Cited by 224 · see citing works
Self-citationMeasuring occupational stress: Development of the Pressure Management Indicator.
Williams, S., & Cooper, C. L. (1998) · Journal of Occupational Health Psychology · doi:10.1037/1076-8998.3.4.306 · Cited by 194 · see citing works
The model human processor: An engineering model of human performance.
Card, S. K., Moran, T. P., & Newell, A. (1986) · Cited by 184 · see citing works
Smith, A. W., McGeeney, K., & Duggan, M. (2015) · Cited by 176 · see citing works
Kreutzer, J. S., DeLuca, J., & Caplan, B. (2010) · Springer eBooks · doi:10.1007/978-0-387-79948-3_6121 · Cited by 168 · see citing works
Hernandez, J., Paredes, P. E., Roseway, A., & Czerwinski, M. P. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557165 · Cited by 167 · see citing works
An exploratory study into deception detection in text-based computer-mediated communication
Zhou, L., Twitchell, D. P., Qin, T., Burgoon, J. K., & Nunamaker, J. F. (2003) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2003.1173793 · Cited by 161 · see citing works
Predicting searcher frustration
Feild, H. A., Allan, J., & Jones, R. (2010) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1835449.1835458 · Cited by 158 · see citing works
Human performance measures handbook
Gawron, V. J. (2000) · Cited by 130 · see citing works
A review of emotion recognition methods based on keystroke dynamics and mouse movements
Kolakowska, A. (2013) · 2013 6th International Conference on Human System Interactions (HSI) · doi:10.1109/hsi.2013.6577879 · Cited by 130 · see citing works
A comparison of two scoring procedures with the NASA task load index in a simulated flight task
Moroney, W. F., Biers, D. W., Eggemeier, F. T., & Mitchell, J. A. (2003) · IEEE National Aerospace and Electronics Conference (NAECON) · doi:10.1109/naecon.1992.220513 · Cited by 128 · see citing works
Multimodal behavioral analysis for non-invasive stress detection
Carneiro, D. R., Castillo, J. C., Nováis, P., Fernández‐Caballero, A., & Neves, J. (2012) · Expert Systems with Applications · doi:10.1016/j.eswa.2012.05.065 · Cited by 124 · see citing works
Adams, P., Rabbi, M., Rahman, T., Matthews, M. L., Voida, A., Gay, G. K., Choudhury, T., & Voida, S. (2014) · International Conference on Pervasive Computing Technologies for Healthcare · doi:10.4108/icst.pervasivehealth.2014.254959 · Cited by 113 · see citing works
Conceptual Issues in Research on Stress and Health
Semmer, N. K., McGrath, J. E., & Beehr, T. A. (2004) · CRC Press eBooks · doi:10.1201/9781420039702.ch1 · Cited by 112 · see citing works
Estrin, D. L. (2014) · Communications of the ACM · doi:10.1145/2580944 · Cited by 109 · see citing works
Handbook of Human Factors Testing and Evaluation
(2001) · CRC Press eBooks · doi:10.1201/b12458 · Cited by 107 · see citing works
Hay, E. L., & Diehl, M. K. (2010) · Psychology and Aging · doi:10.1037/a0018747 · Cited by 96 · see citing works
Villani, M., Tappert, C. C., Ngo, G., Simone, J., Fort, H. S., & Cha, S.-H. (2006) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2006.115 · Cited by 96 · see citing works
Stress and Cognition: A Cognitive Psychological Perspective
Bourne, L. E., & Yaroush, R. A. (2003) · NASA Technical Reports Server (NASA) · Cited by 95 · see citing works
On the Cognitive Status of Pauses in Discourse Production
Schilperoord, J. (2002) · Studies in writing · doi:10.1007/978-94-010-0468-8_4 · Cited by 88 · see citing works
Following linguistic footprints
Zhou, L., & Zhang, D. (2008) · Communications of the ACM · doi:10.1145/1378727.1389972 · Cited by 79 · see citing works
An Empirical Investigation of Deception Behaviorin Instant Messaging
Zhou, L. (2005) · IEEE Transactions on Professional Communication · doi:10.1109/tpc.2005.849652 · Cited by 72 · see citing works
Khawaja, M. A., Chen, F., & Marcus, N. (2013) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2013.860579 · Cited by 70 · see citing works
Mehrabian, A., & Wiener, M. (1966) · Journal of Consulting Psychology · doi:10.1037/h0023813 · Cited by 61 · see citing works
Zhou, L., & Zhang, D. (2006) · Small Group Research · doi:10.1177/1046496405285125 · Cited by 55 · see citing works
Automatic cognitive load detection from speech features
Yin, B., Ruiz, N., Chen, F., & Khawaja, M. A. (2007) · Proceedings of the 19th Australasian conference on Computer-Human Interaction: Entertaining User Interfaces · doi:10.1145/1324892.1324946 · Cited by 55 · see citing works
The Bayes Point Machine for computer-user frustration detection via pressuremouse
Qi, Y., Reynolds, C., & Picard, R. W. (2001) · doi:10.1145/971478.971495 · Cited by 50 · see citing works
Dinges, D., Venkataraman, S., McGlinchey, E., & Metaxas, D. N. (2007) · Acta Astronautica · doi:10.1016/j.actaastro.2006.09.003 · Cited by 43 · see citing works
StressSense: Detecting Stress in Unconstrained Acoustic Environments using Smartphones
Lu, H., Frauendorfer, D., Gática-Pérez, D., Rabbi, M., Mast, M. S., Chittaranjan, G. T., Campbell, A. T., & Choudhury, T. (2012) · Cited by 39 · see citing works
Using language complexity to measure cognitive load for adaptive interaction design
Khawaja, M. A., Chen, F., & Marcus, N. (2010) · Proceedings of the 15th international conference on Intelligent user interfaces · doi:10.1145/1719970.1720024 · Cited by 32 · see citing works
Classifying Text-Based Computer Interactions for Health Monitoring
Vizer, L. M., & Sears, A. (2015) · IEEE Pervasive Computing · doi:10.1109/mprv.2015.85 · Cited by 18 · see citing works
Self-citationDifferent strokes for different folks
Vizer, L. M. (2013) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2468356.2479508 · Cited by 9 · see citing works
Self-citation
Flagged first, then by citations
3 of the 74 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Mandler G, 2010. Thought, memory, and learning: effects of emotional stressThe Handbook of Stress. The Free Press, New York, NY.
- Sweller J, Ayres P, Kalyuga S, 2011. Cognitive Load Theory. Springer, New York, NY.
Cited by
Sorted by citations
A Survey on Measuring Cognitive Workload in Human-Computer Interaction
Kosch, T., Karolus, J., Zagermann, J., Reiterer, H., Schmidt, A., & Woźniak, P. W. (2023) · ACM Computing Surveys · doi:10.1145/3582272 · Cited by 304 · see citing works
“Additionally, keystroke dynamics and linguistic markers of typed text (i.e., language production) were highly related to cognitive workload [34, 245].”
Multimodal data as a means to understand the learning experience
Giannakos, M. N., Sharma, K., Pappas, I. O., Kostakos, V., & Velloso, E. (2019) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.02.003 · Cited by 210 · see citing works
Situationally-Induced Impairments and Disabilities
Wobbrock, J. O. (2019) · Human-computer interaction series · doi:10.1007/978-1-4471-7440-0_5 · Cited by 36 · see citing works
Exposito, M., Hernandez, J., & Picard, R. W. (2018) · International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI Adjunct) · doi:10.1145/3236112.3236132 · Cited by 24 · see citing works
Accuracy and reproducibility of ChatGPT responses to real‐world drug information questions
Khatri, S., Sengul, A., Moon, J., & Jackevicius, C. A. (2025) · JACCP JOURNAL OF THE AMERICAN COLLEGE OF CLINICAL PHARMACY · doi:10.1002/jac5.70038 · Cited by 18 · see citing works
Towards Participant-Independent Stress Detection Using Instrumented Peripherals
Silva, D. R. D. C., Wang, Z., & Gutiérrez‐Osuna, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3061417 · Cited by 17 · see citing works
Twose, J., Licitra, G., McConchie, H., Lam, K., & Killestein, J. (2020) · Chaos An Interdisciplinary Journal of Nonlinear Science · doi:10.1063/5.0022031 · Cited by 16 · see citing works
Polluted cognition: The effect of air pollution on online purchasing behavior
Li, C., Xia, X., Lin, Y., & Wen, H. (2024) · Journal of Retailing and Consumer Services · doi:10.1016/j.jretconser.2024.103839 · Cited by 11 · see citing works
From movement to learning: Leveraging VR behavioral metrics to evaluate cognitive load and curiosity
Poupard, M., Larrue, F., Bertrand, M. M., Liguoro, D., Sauzeon, H., & Tricot, A. (2026) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2026.103751 · Cited by 11 · see citing works
Mulholland, M. M., Stuifbergen, A. M., Schutz, A. D. L. T., Franco‐Rocha, O. Y., Blayney, D. W., & Kesler, S. R. (2024) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2024.1421703 · Cited by 7 · see citing works
Распознавание изменений психофизиологического состояния по характеристикам печати на клавиатуре
Геннадьевна, Х. Ю., Катаева, Г. В., Мещеряков, А., Крестовоздвиженский, С., Григорьевна, С. М., & Бойцова, Ю. (2024) · Психологические исследования · doi:10.54359/ps.v17i95.1594
- Authors
- Vizer, L. M., & Sears, A.
- Affiliations
- University of Maryland, Baltimore County (US)
- Corresponding author
- Vizer, Lisa M.
- ORCID
- Vizer, Lisa M.
- Venue
- International Journal of Human-Computer Studies
- Publisher
- Elsevier BV
- Published
- 2017-08
- Volume
- 104
- Pages
- 80-96
- ISSN
- 1071-5819 (ISSN-L)
- PMC
- PMC8153188
- OpenAlex
- W2593362178
- Cited by
- 12
- References
- 74
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 4 · 2023: 1 · 2021: 1 · 2020: 1 · 2019: 2 · 2018: 1
- FWCI
- 1.42
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- www.elsevier.com/open-access/userlicense/1.0/ (am, 248 d)
- doi.org/10.15223/policy-017 (stm-asf)
- doi.org/10.15223/policy-037 (stm-asf)
- doi.org/10.15223/policy-012 (stm-asf)
- doi.org/10.15223/policy-029 (stm-asf)
- doi.org/10.15223/policy-004 (stm-asf)
- Locations
- International Journal of Human-Computer Studies (acceptedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Science Foundation Graduate Research Fellowship
- National Institutes of Health
- National Library of Medicine (NLM) Biomedical and Health Informatics Training Program at the University of Washington (T15LM007442)
- National Institutes of Health National Center for Advancing Translational Sciences (NCATS) (UL1TR001111)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Keywords
- cognitive demand, keystroke dynamics, text-based interaction, personalized models, individual differences, lexical diversity, word length, word count, human-computer interaction, personalized health management
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
