CoChat

A Survey on Measuring Cognitive Workload in Human-Computer Interaction

Open access

The DOI matches its Crossref record; 267 of the 271 references it cites were found in a registry or an index, and 2 are not checked yet; 16 of the 76 sources its claims cite were checked for support.

Abstract

The ever-increasing number of computing devices around us results in more and more systems competing for our attention, making cognitive workload a crucial factor for the user experience of human-computer interfaces. Research in Human-Computer Interaction (HCI) has used various metrics to determine users’ mental demands. However, there needs to be a systematic way to choose an appropriate and effective measure for cognitive workload in experimental setups, posing a challenge to their reproducibility. We present a literature survey of past and current metrics for cognitive workload used throughout HCI literature to address this challenge. By initially exploring what cognitive workload resembles in the HCI context, we derive a categorization supporting researchers and practitioners in selecting cognitive workload metrics for system design and evaluation. We conclude with three following research gaps: (1) defining and interpreting cognitive workload in HCI, (2) the hidden cost of the NASA-TLX, and (3) HCI research as a catalyst for workload-aware systems, highlighting that HCI research has to deepen and conceptualize the understanding of cognitive workload in the context of interactive computing systems.

Verified 2026-11-19 · 2 references not checked yet

Overview

CoChat Summary

TL;DR

This ACM Computing Surveys paper systematically reviews how cognitive workload has been measured in HCI, drawing on a final corpus of 579 records. It categorizes subjective, physiological, behavioral, and performance-based metrics and proposes a step-by-step procedure for selecting suitable measures, while identifying three research gaps around definition, NASA-TLX reliance, and workload-aware systems.

Key findings

  • The review’s final corpus contains 579 records: 482 from 26 conferences and 97 from 11 journals, derived from an initial corpus of 6,414 papers (4,408 conference, 2,006 journal).
  • Questionnaires are the most used metric, with the NASA-TLX described as the most frequently used questionnaire in the corpus; the paper argues its dominance may reflect community convention and an “academic tradition” rather than demonstrated HCI-specific validity.
  • The authors identify three research gaps: (1) defining and interpreting cognitive workload in HCI, (2) the hidden cost of the NASA-TLX, and (3) HCI research as a catalyst for workload-aware systems.
  • They propose a four-step procedure for choosing cognitive workload metrics, covering whether cognitive workload is the right concept, whether it varies during use, whether the system must adapt to it, and which modality to choose.

Methods

  • Systematic literature review following PRISMA procedures, with a keyword search in HCI-related proceedings and journals; venues included the top 20 conferences plus six conferences in the top 40, and all top 10 journals, confirmed via CORE 4 rankings, with a final cut-off date of 30 April 2020.
  • The search used a regular expression matching terms such as “cognitive/mental load/workload” and “working memory”; the initial corpus was 6,414 papers.
  • Filtering used paper eligibility (archival, peer-reviewed papers only; 904 excluded) and relevance via a double decision scheme with four researchers; 4,931 papers were excluded at relevance, yielding 579 records. Categorization used qualitative coding, a naming scheme agreed by four researchers, and affinity diagramming to build a taxonomy of measures.

Limitations

  • The paper states that its qualitative analysis cannot guarantee the proposed taxonomy is the only correct one, though it aims for comprehensiveness.
  • The paper states that the final choice of cognitive workload modality is almost always a trade-off between ecological validity and the requirements of the measurement modality, and that NASA-TLX validity in HCI is unclear because no empirical evidence determines the range of artifacts or experiences to which it applies.
  • Evident from the text: the search is limited to selected top HCI venues and a cut-off of 30 April 2020, and many irrelevant papers were excluded due to ambiguity of the term “working memory,” especially in computer science venues.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3582272

Cites

267 of 271 references verified0 citation errors · 2 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 PRISMAflow chart highlighting the stages of our systematic search as detailed in Sections 3.1 and 3.2.
Fig. 1 PRISMAflow chart highlighting the stages of our systematic search as detailed in Sections 3.1 and 3.2.CC-BY · From the PDF
Image 2
CC-BY · From the PDF
Image 3
CC-BY · From the PDF
Image 4
CC-BY · From the PDF
Image 5
CC-BY · From the PDF

Verification

Claims the paper makes

6 Valid · 16 Not valid · 23 Not checked · 86 Own results and interpretations

52 more claims · 4 with their cited sources checked

Create a free account to see all 131 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 35
  • Probably not a substantial claim. 49

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Items 201–271 of 271

Research works printed but not linked: 9

  1. 10.5555/2993778.2993801 · doi:10.5555/2993778.2993801
  2. S. G. Hart, M. E. Childress, and J. R. Hauser. 1982. Individual definitions of the term “workload”. In Eighth Symposium on Psychology in the Department of Defense. 478–485.
  3. Sandra G. Hart. 1986. Theory and measurement of human workload. Human Productivity Enhancement 1 (1986), 396–456.
  4. A. V. Korshakov, A. A. Frolov, and P. D. Bobrov. 2010. On-line automatic suppression of artifacts in multi-dimensional signals using ICA. In Abstr. 10th Europ. Conf. on Non-Destructive Testing. 370.
  5. Thomas Kosch and Lewis L. Chuang. 2019. Investigating the influence of RSVP display parameters on working memory load using electroencephalography. In 2nd International Conference on Neuroadaptive Technology.
  6. Gerald Matthews, James Szalma, April Rose Panganiban, Catherine Neubauer, and Joel S. Warm. 2013. Profiling task stress with the Dundee Stress State Questionnaire. Psychology of Stress: New Research 1 (2013), 49–90.
  7. Christopher D. Wickens. 1984. Processing resources in attention. Varieties of Attention (1984).
  8. 10.5555/2993778.2993814 · doi:10.5555/2993778.2993814
  9. F. R. H. Zijlstra and L. Van Doorn. 1985. The Construction of a Scale to Measure Perceived Effort. University of Technology.

Cited by

See all 304

Metadata

Authors
Kosch, T., Karolus, J., Zagermann, J., Reiterer, H., Schmidt, A., & Woźniak, P. W.
Affiliations
  • Humboldt-Universität zu Berlin (DE)
  • German Research Centre for Artificial Intelligence (DE)
  • University of Konstanz (DE)
  • Ludwig-Maximilians-Universität München (DE)
  • Chalmers University of Technology (SE)
Venue
ACM Computing Surveys
Published
2023-07-13
Volume
55
Issue
13s
Pages
1-39
ISSN
  • 0360-0300 (ISSN-L)
  • 1557-7341
OpenAlex
W4318618520
Cited by
304
References
259
Citations by year
2026: 120 · 2025: 95 · 2024: 64 · 2023: 25
FWCI
89.72
Type
journal article
Language
en
Open access
Locations
  • ACM Computing Surveys (publishedVersion, cc-by)
  • Chalmers Research (Chalmers University of Technology) (submittedVersion)
Funders
Deutsche Forschungsgemeinschaft (251654672 – TRR 161 and 414984028 – CRC 1404 FONDA)
Topics
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
Keywords
mental workload, workload metrics, NASA-TLX, mental demand, user experience, reproducibility, interactive systems

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.