A Survey on Measuring Cognitive Workload in Human-Computer Interaction
Open accessThe DOI matches its Crossref record; 267 of the 271 references it cites were found in a registry or an index, and 2 are not checked yet; 16 of the 76 sources its claims cite were checked for support.
Abstract
The ever-increasing number of computing devices around us results in more and more systems competing for our attention, making cognitive workload a crucial factor for the user experience of human-computer interfaces. Research in Human-Computer Interaction (HCI) has used various metrics to determine users’ mental demands. However, there needs to be a systematic way to choose an appropriate and effective measure for cognitive workload in experimental setups, posing a challenge to their reproducibility. We present a literature survey of past and current metrics for cognitive workload used throughout HCI literature to address this challenge. By initially exploring what cognitive workload resembles in the HCI context, we derive a categorization supporting researchers and practitioners in selecting cognitive workload metrics for system design and evaluation. We conclude with three following research gaps: (1) defining and interpreting cognitive workload in HCI, (2) the hidden cost of the NASA-TLX, and (3) HCI research as a catalyst for workload-aware systems, highlighting that HCI research has to deepen and conceptualize the understanding of cognitive workload in the context of interactive computing systems.
Verified 2026-11-19 · 2 references not checked yet
Overview
CoChat Summary
TL;DR
This ACM Computing Surveys paper systematically reviews how cognitive workload has been measured in HCI, drawing on a final corpus of 579 records. It categorizes subjective, physiological, behavioral, and performance-based metrics and proposes a step-by-step procedure for selecting suitable measures, while identifying three research gaps around definition, NASA-TLX reliance, and workload-aware systems.
Key findings
- The review’s final corpus contains 579 records: 482 from 26 conferences and 97 from 11 journals, derived from an initial corpus of 6,414 papers (4,408 conference, 2,006 journal).
- Questionnaires are the most used metric, with the NASA-TLX described as the most frequently used questionnaire in the corpus; the paper argues its dominance may reflect community convention and an “academic tradition” rather than demonstrated HCI-specific validity.
- The authors identify three research gaps: (1) defining and interpreting cognitive workload in HCI, (2) the hidden cost of the NASA-TLX, and (3) HCI research as a catalyst for workload-aware systems.
- They propose a four-step procedure for choosing cognitive workload metrics, covering whether cognitive workload is the right concept, whether it varies during use, whether the system must adapt to it, and which modality to choose.
Methods
- Systematic literature review following PRISMA procedures, with a keyword search in HCI-related proceedings and journals; venues included the top 20 conferences plus six conferences in the top 40, and all top 10 journals, confirmed via CORE 4 rankings, with a final cut-off date of 30 April 2020.
- The search used a regular expression matching terms such as “cognitive/mental load/workload” and “working memory”; the initial corpus was 6,414 papers.
- Filtering used paper eligibility (archival, peer-reviewed papers only; 904 excluded) and relevance via a double decision scheme with four researchers; 4,931 papers were excluded at relevance, yielding 579 records. Categorization used qualitative coding, a naming scheme agreed by four researchers, and affinity diagramming to build a taxonomy of measures.
Limitations
- The paper states that its qualitative analysis cannot guarantee the proposed taxonomy is the only correct one, though it aims for comprehensiveness.
- The paper states that the final choice of cognitive workload modality is almost always a trade-off between ecological validity and the requirements of the measurement modality, and that NASA-TLX validity in HCI is unclear because no empirical evidence determines the range of artifacts or experiences to which it applies.
- Evident from the text: the search is limited to selected top HCI venues and a cut-off of 30 April 2020, and many irrelevant papers were excluded due to ambiguity of the term “working memory,” especially in computer science venues.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
6 Valid · 16 Not valid · 23 Not checked · 86 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Previous research has shown that the size and speed of saccades are highly discriminatory parameters for an increased cognitive workload: Here, an increased cognitive workload is indicated by larger or faster saccades [47].”
§ A Survey on Measuring Cognitive Workload in Human-Computer Interaction > 4.3.2 Saccades.
What the sources say
- Disputedcites [47] 47. Siyuan Chen, Julien Epps, Natalie Ruiz, and Fang Chen. 2011. Eye activity as a measure of human mental effort in HCI. In Proceedings of the 16th International Conference on Intelligent User Interfaces (IUI'11). ACM, New York, NY, USA, 315-318. https://doi.org/10.1145/1943403.1943454 sourceExperimental — accuracy not yet measured
The Discussion says saccade speed and especially size were highly discriminatory, but reports that both decreased as the task demanded more working memory and attention. The cited paper therefore supports their usefulness as measures, but gives the opposite direction from the claim’s ‘larger or faster’ saccades. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“Similarly, touch input [39] -especially finger trajectories [173] -were used to measure cognitive workload.”
§ A Survey on Measuring Cognitive Workload in Human-Computer Interaction > 4.7 Haptic Interaction
What the sources say
- Not checked — no full text of the sourcecites [39] 39. Davide Maria Calandra, Antonio Caso, Francesco Cutugno, Antonio Origlia, and Silvia Rossi. 2013. CoWME: A gen- eral framework to evaluate cognitive workload during multimodal interaction. In Proceedings of the 15th ACM on International Conference on Multimodal Interaction (ICMI'13). ACM, Sydney, Australia, 111-118. https://doi.org/10. 1145/2522848.2522867 source
- Disputedcites [173] 173. E. M. Peck, E. Carlin, and R. Jacob. 2015. Designing brain-computer interfaces for attention-aware systems. Computer 48, 10 (Oct. 2015), 34-42. https://doi.org/10.1109/MC.2015.315 sourceExperimental — accuracy not yet measured
The citation [173] is attached specifically to “finger trajectories.” In “Training CARSON to Read the Brain” and Figure 3 (page not provided), the paper instead describes predicting cognitive workload from fNIRS brain-sensing signals, not finger trajectories or touch input. (LLM, conf=0.85)
Checked by a reasoning model — may be wrong.
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Similarly, beta power is widely used as a metric for mental workload, including alertness, arousal, frustration, engagement, and workload states [141].”
§ A Survey on Measuring Cognitive Workload in Human-Computer Interaction > 4.2.1 Electroencephalography (EEG).
What the sources say
- Could not verifycites [141] 141. Kristiyan Lukanov, Horia A. Maior, and Max L. Wilson. 2016. Using fNIRS in Usability Testing: Understanding the Effect of Web Form Layout on Mental Workload. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI'16). ACM, Santa Clara, California, USA, 4011-4016. https://doi.org/10.1145/2858036.2858236 sourceExperimental — accuracy not yet measured
The Measuring Mental Workload and Objective Measures sections discuss psychophysiological measures and fNIRS haemoglobin measurements, but do not address beta power. Searches for beta power and EEG spectral power found no passage linking beta power to alertness, arousal, frustration, engagement, or workload states. The available passages have no page numbers. (LLM, conf=0.76)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
Every source the paper cites for this claim was checked and supports it.
In the paper
“Previous research shows that the rate and duration of fixation are indicators of an attention shift due to increasing task complexity [47].”
§ A Survey on Measuring Cognitive Workload in Human-Computer Interaction > 4.3.4 Fixations.
What the sources say
- Supportedcites [47] 47. Siyuan Chen, Julien Epps, Natalie Ruiz, and Fang Chen. 2011. Eye activity as a measure of human mental effort in HCI. In Proceedings of the 16th International Conference on Intelligent User Interfaces (IUI'11). ACM, New York, NY, USA, 315-318. https://doi.org/10.1145/1943403.1943454 source
The Discussion reports that fixation duration and fixation rate changed with task complexity and interprets those results as indicating that significantly more attention was needed for the more complex task. It also specifies that fixation duration increased while fixation rate decreased as attentional demands rose. (LLM, conf=0.92)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Previous research has shown that pupils dilate with increasing task difficulty [47] across tasks like reading, problem-solving, or visual search activities [208], where the cognitive demand can be estimated in real-time using machine learning [131].”
§ A Survey on Measuring Cognitive Workload in Human-Computer Interaction > 4.3.1 Pupil Diameter.
What the sources say
- Partly supportedcites [47] 47. Siyuan Chen, Julien Epps, Natalie Ruiz, and Fang Chen. 2011. Eye activity as a measure of human mental effort in HCI. In Proceedings of the 16th International Conference on Intelligent User Interfaces (IUI'11). ACM, New York, NY, USA, 315-318. https://doi.org/10.1145/1943403.1943454 sourceExperimental — accuracy not yet measured
The Discussion reports increased pupil size when participants performed the more demanding visual training task, supporting the link between pupil dilation and task difficulty. The paper does not establish that pattern across the cited examples of reading, problem-solving, and visual search; its own task involved recalling player positions in a basketball video. (LLM, conf=0.86)
- Not checked — no full text of the sourcecites [208] 208. Namrata Srivastava, Eduardo Velloso, Jason M. Lodge, Sarah Erfani, and James Bailey. 2019. Continuous evaluation of video lectures from real-time difficulty self-report. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI'19). Association for Computing Machinery, Glasgow, Scotland, UK, 1-12. https://doi.org/ 10.1145/3290605.3300816 source
- Could not verifycites [131] 131. Moira LeMay and Eric Hird. 1986. Operator work load: When is enough enough? Commun. ACM 29, 7 (July 1986), 638-642. https://doi.org/10.1145/6138.6147 sourceExperimental — accuracy not yet measured
The Results report that time-stress ratings increased with workload, and the introduction lists physiological measures as a category, but neither addresses pupil dilation. Searches for pupil diameter, task difficulty, and pupillary response found no passage supporting the claim attributed to this paper; the task examples are attached to a different citation. (LLM, conf=0.75)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
52 more claims · 4 with their cited sources checked
Create a free account to see all 131 claims and every source verdict.
- It may state two separate claims as one. 35
- Probably not a substantial claim. 49
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Survey on Measuring Cognitive Workload in Human-Computer Interaction > 4.1 Questionnaires
“Questionnaires lack real-time capabilities since they are employed after an experimental condition.”
§ A Survey on Measuring Cognitive Workload in Human-Computer Interaction > 4.1.3 Instantaneous Self-Assessment.
“Participants rate their perceived workload in predefined intervals during the execution of a task -allowing for real-time assessment of cognitive workload.”
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · cited 1×
“The first phase of our review was a systematic keyword search in the digital library databases relevant to the HCI field.”
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003 · cited 5×
“The NASA Task Load Index (NASA-TLX) [90, 91] is a multi-scale questionnaire to assess participants’ perceived subjective workload.”
InfluentialCognitive Load During Problem Solving: Effects on Learning
Sweller, J. (1988) · Cognitive Science · doi:10.1016/0364-0213(88)90023-7 · Cited by 11,048 · cited 3×
“Sweller developed the cognitive load theory [231–233], a theoretical framework that explains how learning is limited through instructional design.”
SUS: A 'Quick and Dirty' Usability Scale
(1996) · Usability Evaluation In Industry · doi:10.1201/9781498710411-35 · Cited by 8,989
Brain–computer interfaces for communication and control
Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., & Vaughan, T. M. (2002) · Clinical Neurophysiology · doi:10.1016/s1388-2457(02)00057-3 · Cited by 7,927 · cited 1×
“Participants rate their perceived work-load in predefined intervals during the execution of a task – allowing for real-time assessment of cognitive workload.”
Cognitive Architecture and Instructional Design
Sweller, J., van Merrienboer, J. J. G., & Paas, F. G. W. C. (1998) · Educational Psychology Review · doi:10.1023/a:1022193728205 · Cited by 6,023 · cited 3×
“…workload helps interface designers in two ways: using a constructive approach (e.g., during early design phases), where cognitive workload is measured to improve the design of novel interactive systems, or using a summative approach (e.g., using existing systems) to compare the cognitive…”
(2005) · Cited by 5,977
Systematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537 · cited 1×
“We followed the PRISMA procedures [121, 174] for systematic reviews and meta-analyses.”
(2001) · doi:10.1017/cbo9780511546396 · Cited by 4,482
NASA-Task Load Index (NASA-TLX); 20 Years Later
Hart, S. G. (2006) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120605000909 · Cited by 4,297 · cited 3×
“The NASA-TLX [90, 91] questionnaire has become a household name in HCI research, and it is widely taught in HCI education.”
Research works printed but not linked: 9
- 10.5555/2993778.2993801 · doi:10.5555/2993778.2993801
- S. G. Hart, M. E. Childress, and J. R. Hauser. 1982. Individual definitions of the term “workload”. In Eighth Symposium on Psychology in the Department of Defense. 478–485.
- Sandra G. Hart. 1986. Theory and measurement of human workload. Human Productivity Enhancement 1 (1986), 396–456.
- A. V. Korshakov, A. A. Frolov, and P. D. Bobrov. 2010. On-line automatic suppression of artifacts in multi-dimensional signals using ICA. In Abstr. 10th Europ. Conf. on Non-Destructive Testing. 370.
- Thomas Kosch and Lewis L. Chuang. 2019. Investigating the influence of RSVP display parameters on working memory load using electroencephalography. In 2nd International Conference on Neuroadaptive Technology.
- Gerald Matthews, James Szalma, April Rose Panganiban, Catherine Neubauer, and Joel S. Warm. 2013. Profiling task stress with the Dundee Stress State Questionnaire. Psychology of Stress: New Research 1 (2013), 49–90.
- Christopher D. Wickens. 1984. Processing resources in attention. Varieties of Attention (1984).
- 10.5555/2993778.2993814 · doi:10.5555/2993778.2993814
- F. R. H. Zijlstra and L. Van Doorn. 1985. The Construction of a Scale to Measure Perceived Effort. University of Technology.
Cited by
Digital deception: generative artificial intelligence in social engineering and phishing
Schmitt, M., & Flechais, I. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10973-2 · Cited by 198
“The expectation that individuals can scrutinize every email they receive is impractical and counterproductive, especially in today’s fast-paced digital environment, that comes with heavy cognitive workload (Kosch et al. 2023 ).”
Industrial applications of large language models
Raza, M., Jahangir, Z., Riaz, M. B., Saeed, M. J., & Sattar, M. A. (2025) · Scientific Reports · doi:10.1038/s41598-025-98483-1 · Cited by 163
Workplace Well-Being in Industry 5.0: A Worker-Centered Systematic Review
Antonaci, F. G., Olivetti, E. C., Marcolin, F., Jiménez, I. A. C., Eynard, B., Vezzetti, E., & Moos, S. (2024) · Sensors · doi:10.3390/s24175473 · Cited by 121
“In recent years, there has been a proliferation of studies seeking the best physiological parameters to characterize and detect these emotional states [89,90], but most of these have been carried out in the laboratory under controlled conditions.”
BackgroundChiossi, F., Turgut, Y., Welsch, R., & Mayer, S. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3604243 · Cited by 59
“The idea is that physiological arousal can be used as a proxy for cognitive load [21, 64].”
BackgroundChiossi, F., Ou, C., Gerhardt, C., Putze, F., & Mayer, S. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103433 · Cited by 75
A survey of explainable knowledge tracing
Bai, Y., Zhao, J., Wei, T., Cai, Q., & He, L. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05509-8 · Cited by 75
Faudzi, M. A., Cob, Z. C., Ghazali, M., Omar, R., & Sharudin, S. A. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e37494 · Cited by 88
“It represents the connection between the visual presentation of a task and the resulting workload [54].”
BackgroundMethodDigital Deception: Generative Artificial Intelligence in Social Engineering and PhishingPreprint
Schmitt, M., & Fléchais, I. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4602790 · Cited by 37
Integration of IoT and blockchain for decentralized management and ownership in the metaverse
Din, I. U., Awan, K. A., Almogren, A., & Rodrigues, J. J. P. C. (2023) · International Journal of Communication Systems · doi:10.1002/dac.5612 · Cited by 35
“8 Current interpretations envision the metaverse as an interactive, immersive, and shared virtual domain serving as a digital extension of our physical world.”
BackgroundKazemitabaar, M., Huang, O., Suh, S., Henley, A. Z., & Grossman, T. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712104 · Cited by 92
Kloft, A. M., Welsch, R., Kosch, T., & Villa, S. (2023) · arXiv · doi:10.48550/arxiv.2309.16606 · Cited by 34
Self-citationHypergraph-Based Multi-View Action Recognition Using Event Cameras
Gao, Y., Lu, J., Li, S., Li, Y., & Du, S. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3382117 · Cited by 52
“It has gained significant attention due to its broad applications in various fields, including surveillance, human-computer interaction, robotics, and video content analysis [1], [2], [3], [4].”
BackgroundKloft, A. M., Welsch, R., Kosch, T., & Villa, S. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642633 · Cited by 53
“To test H1 and H2 with respect to workload, we implemented the NASA-TLX [29], a well-established questionnaire [39], with six dimensions: mental demand, physical demand, temporal demand, performance, effort, and frustration.”
Self-citationSun, C., Zhao, X., Guo, B., & Chen, N. (2025) · Behavioral Sciences · doi:10.3390/bs15050648 · Cited by 55
“In contrast, excessive workload, especially in cognitively demanding roles, can limit the mental bandwidth necessary for exploring new opportunities and hinder proactive engagement (Kosch et al., 2023).”
Babaei, E., Dingler, T., Tag, B., & Velloso, E. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103515 · Cited by 64
HUMAN-COMPUTER INTERACTION: ENHANCING USER EXPERIENCE IN INTERACTIVE SYSTEMS
Kheder, H. A. (2023) · Kufa Journal of Engineering · doi:10.30572/2018/kje/140403 · Cited by 27
A robust operators’ cognitive workload recognition method based on denoising masked autoencoder
Yu, X., & Chen, C. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112370 · Cited by 29
TicTacToes: Assessing Toe Movements as an Input Modality
Müller, F., Schmitt, D., Matviienko, A., Schön, D., Günther, S., Kosch, T., & Schmitz, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580954 · Cited by 24
“Besides the technical limitations, questions remain on how to handle external factors, such as the cognitive demand of the interaction [49], the shape, or cut of the footwear.”
Self-citationGonnermann-Müller, J., & Krüger, J. M. (2024) · Journal of Computer Assisted Learning · doi:10.1111/jcal.13095 · Cited by 43
SensCon: Embedding Physiological Sensing into Virtual Reality Controllers
Chiossi, F., Kosch, T., Menghini, L., Villa, S., & Mayer, S. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3604270 · Cited by 26
“This would allow us to understand better how SensCon performs in scenarios that involve more erratic and everchanging movements with varied cognitive load [61] Ultimately, we”
BackgroundSelf-citationJeong, S., Kim, J., & Lee, J. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3372126 · Cited by 29
“Cognitive load arises when external events impose demands on individuals’ mental resources within a specific context [28].”
BackgroundHelping the Helper : Supporting Peer Counselors via AI-Empowered Practice and Feedback
Hsu, S.-L., Shah, R. S., Senthil, P., Ashktorab, Z., Dugan, C., Geyer, W., & Yang, D. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710993 · Cited by 112
A comprehensive review of extended reality and its application in aerospace engineering
Rostami, M., Pradhan, P., Karki, N., Omorodion, J., Milani, P., Kamoonpuri, J. M., Liu, H. Y., & Chung, J. (2025) · Progress in Aerospace Sciences · doi:10.1016/j.paerosci.2025.101118 · Cited by 26
A Comparison of Virtual Reality Menu Archetypes: Raycasting, Direct Input, and Marking Menus
Wentzel, J., Lakier, M., Hartmann, J., Shazib, F., Casiez, G., & Vogel, D. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3420236 · Cited by 27
“Single Level Menus for Usability and Accuracy: If subjective usability is paramount, raycast menus offer the most consistent high ratings regardless of number of items.”
Feature-Enhanced Multimodal Interaction model for emotion recognition in conversation
Fu, Y., Yan, X., Chen, W., & Zhang, J. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112876 · Cited by 22
Tailor Twist: Assessing Rotational Mid-Air Interactions for Augmented Reality
Schön, D., Kosch, T., Müller, F., Schmitz, M., Günther, S., Bommhardt, L., & Mühlhäuser, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581461 · Cited by 20
“Still, we must consider the infuence of physical and cognitive fatigue [36] on our fndings.”
BackgroundSelf-citationVisual Grounding in 2D and 3D: A unified perspective and survey
Guo, K., Huang, Y., Jia, T., Song, X., Sun, S., Wei, H., Han, X.-F., Huang, S., Strisciuglio, N., & Li, S. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103625 · Cited by 37
Exploring Mixed Reality in General Aviation to Support Pilot Workload
Katins, C., Feger, S. S., & Kosch, T. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585742 · Cited by 15
“The NASA-TLX [9] was chosen to gain insights into the assessment of the perceived task load by the participants [14].”
MethodSelf-citationKatins, C., Woźniak, P. W., Chen, A., Tumay, I., Le, L., Uschold, J., & Kosch, T. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642631 · Cited by 23
Self-citationA Systematic Literature Review: Cognitive Workload Assessment in Human Factors Research
Tierney, M., Adeyemi, E. O., & Bommer, S. (2025) · Advances in Human-Computer Interaction · doi:10.1155/ahci/9313239 · Cited by 17
“Cognitive workload is a theoretical framework for predicting how much mental efort will be required to complete a task within a specifc period under certain conditions [1, 5, 6, 13].”
A review on cognitive workload for industry 5.0
Vitti, M., Padovano, A., & Facchini, F. (2025) · Computers & Industrial Engineering · doi:10.1016/j.cie.2025.111350 · Cited by 23
Li, Y., Tag, B., Dai, S., Crowther, R. G., Dwyer, T., Irani, P., & Ens, B. M. (2024) · ACM Transactions on Graphics · doi:10.1145/3658230 · Cited by 17
“Second, participants with diverse backgrounds may interpret and apply the scales differently and introduce potential bias to the results [Borg 1998; Kosch et al. 2023].”
BackgroundZheng, H., Xiao, D., & Zhou, J. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2350839 · Cited by 24
“User participation often stems from satisfaction with the user experience, and this satisfaction is significantly influenced by HCI quality (Chiang et al., 2020; Kosch et al., 2023).”
Sänger, M., Mecquenem, N. D., Lewińska, K. E., Bountris, V., Lehmann, F., Leser, U., & Kosch, T. (2024) · GigaScience · doi:10.1093/gigascience/giae030 · Cited by 23
“Parallel to this, there is a well-established field in human-computer interaction focusing on assisting end-user programmers and software development, as highlighted by previous work [8, 9, 10, 11], to reduce the perceived cognitive workload and improve the overall programmer performance [12].”
BackgroundSelf-citationPi, Z., Zhang, Y., Yu, Q., & Yang, J. (2023) · Educational Technology Research and Development · doi:10.1007/s11423-023-10256-7 · Cited by 14
“Other surveys have explored measuring cognitive workload in the domain of human–computer interaction (Kosch et al., 2023), while some have developed and validated theory-based questionnaires for different types of cognitive load (Krieglstein et al., 2023).”
BackgroundNext Generation Computing and Communication Hub for First Responders in Smart Cities
Shaposhnyk, O. A., Lai, K., Wolbring, G., Shmerko, V. P., & Yanushkevich, S. N. (2024) · Sensors · doi:10.3390/s24072366 · Cited by 18
Caro‐Álvaro, S., García‐López, E., Brun-Guajardo, A., García‐Cabot, A., & Mavri, A. (2024) · Sensors · doi:10.3390/s24031004 · Cited by 21
“Gesture-based interactions could be seen as non-regular This clearly demonstrates the opportunities of integrating accelerometers and gyro-scopes in the development of mobile applications. when engaging with novel tasks (as derived from Banu, Al Siyabi and Al Minje [24 and Kosch et al. [25]).”
MethodComparing head-mounted and handheld augmented reality for guided assembly
Leins, N., Gonnermann-Müller, J., & Teichmann, M. R. (2024) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-024-00440-1 · Cited by 27
Usmani, U. A., & Usmani, M. U. (2023) · International Conference on Ambient Intelligence, Knowledge Informatics and Industrial Electronics (AIKIIE) · doi:10.1109/aikiie60097.2023.10390262 · Cited by 13
Federated Learning for Privacy-aware Cognitive Workload Estimation
Fenoglio, D., Josifovski, D., Gobbetti, A., Formo, M., Gjoreski, H., Gjoreski, M., & Langheinrich, M. (2023) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3626705.3627783 · Cited by 16
Huang, Y.-T., Hsu, C.-C., & Wang, T.-H. (2025) · Frontiers in Virtual Reality · doi:10.3389/frvir.2025.1540406 · Cited by 19
“Additionally, Kosch et al. (2023) suggested that the widespread use of NASA-TLX in HCI applications may be more a result of historical precedent and convenience rather than because it is the optimal measurement tool.”
InfluentialLuo, S., Zhang, C., Zhu, W., et al. (2024) · Signal Image and Video Processing · doi:10.1007/s11760-024-03619-x · Cited by 11
“Pilots’ assessments may be influenced by their emotional states, individual differences, and personal biases, potentially leading to less objective results [ 49 ].”
Relaxed forced choice improves performance of visual quality assessment methods
Jenadeleh, M., Zagermann, J., Reiterer, H., et al. (2023) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex58391.2023.10178467 · Cited by 10
“The analysis of the overall TLX scores did not show statistically significant differences (z = −1.65, p > .05) with median values of 57.50 for AFC and 57.50 for RFC.”
MethodInfluentialSelf-citationGrootjen, J. W., Thalhammer, P., & Kosch, T. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3676531 · Cited by 14
Self-citationEvaluating Typing Performance in Different Mixed Reality Manifestations using Physiological Features
Chiossi, F., Khaoudi, Y. E., Ou, C., Sidenmark, L., Zaky, A., Feuchtner, T., & Mayer, S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3698142 · Cited by 13
Gerling, K., Meiners, A.-L., Schumm, L., Rixen, J. O., Wolf, M., Yıldız, Z., Alexandrovsky, D., & Opp, M. S. (2025) · ACM Transactions on Accessible Computing · doi:10.1145/3770755 · Cited by 30
Revisiting Consumed Endurance: A NICE Way to Quantify Shoulder Fatigue in Virtual Reality
Li, Y., Crowther, R. G., Smiley, J., et al. (2023) · ACM Symposium on Virtual Reality Software and Technology (VRST) · doi:10.1145/3611659.3615702 · Cited by 9
“The second drawback is that individuals may perceive the scales differently due to the uniqueness of their backgrounds and bring bias to study findings [6, 23].”
BackgroundA Systematic Literature Review to Characterize Asymmetric Interaction in Collaborative Systems
Bréhault, V., Dubois, E., Prouzeau, A., & Serrano, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713129 · Cited by 15
Ayub, A., Francesco, Z. D., Holthaus, P., Nehaniv, C. L., & Dautenhahn, K. (2025) · International Journal of Social Robotics · doi:10.1007/s12369-025-01214-9 · Cited by 26
“Finally, for autonomous robots that learn through human teaching, it is also imperative to understand human perceptions of task load for interacting with and teaching the robot [47, 48].”
Sharmin, S., Bakhshipour, E., Abrar, M. F., Kiafar, B., Kullu, P., Getchell, N., & Barmaki, R. (2025) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3716553.3750811 · Cited by 11
Visual cognition-based optimised design of primary flight displays in cockpits
Zeng, Z., Sun, Y., Liu, X., Jie, Y., & Zeng, Y. (2024) · The Aeronautical Journal · doi:10.1017/aer.2024.103 · Cited by 10
Shafiei, S. B., Shadpour, S., & Mohler, J. L. (2024) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208241285513 · Cited by 12
“This relationship is expected in workload studies where high cognitive or physical workload impairs task performance (Galy et al., 2012; Kosch et al., 2023).”
Rodrigues, L. S., Schmidt, T. T., Israel, J. H., Zachow, S., Nyakatura, J. A., & Kosch, T. (2024) · Nordic Conference on Human-Computer Interaction · doi:10.1145/3679318.3685349 · Cited by 14
Self-citationmmDigit: A Real-Time Digit Recognition Framework in Air-Writing Using FMCW Radar
Tian, J., Zou, Y., Lai, J., & Liu, F. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3507369 · Cited by 8
Tran, T., Nguyen, Q. M., Luu, T., Tran, M., Kua, J., Hoang, T. N., & Dien, M. (2024) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2024.100743 · Cited by 16
EEG Features to Quantify the NASA-TLX Factors of Cognitive Workload
Zenia, N. Z., Tarng, S., Dizaji, L. G., & Hu, Y. (2025) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2025.3546515 · Cited by 13
“Notably, the NASA-TLX has become the predominant measure of the CWL in HMS [5].”
Understanding the User Perception and Experience of Interactive Algorithmic Recourse Customization
Koh, S., Kim, B. H., & Jo, S. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3674503 · Cited by 12
Broadening the mind: how emerging neurotechnology is reshaping HCI and interactive system design
Schneegass, C., Wilson, M. L., Shaban, J., Niess, J., Chiossi, F., Mitrevska, T., & Woźniak, P. W. (2024) · i-com · doi:10.1515/icom-2024-0007 · Cited by 11
“23,24 Performance and behavioural measures are a valuable way of assessing people’s capacity to complete tasks, often analysed through observing performance drops or mistakes.”
Self-citationMin, C., Jin, Q., Zhou, J., & Luo, X. (2025) · International Journal of Social Robotics · doi:10.1007/s12369-025-01255-0 · Cited by 14
Implicit Motion State Modeling for Efficient and Effective Video-Level Object Tracking
Jia, X., Hou, Z., Ma, S., & Pu, L. (2026) · Expert Systems with Applications · doi:10.1016/j.eswa.2026.132083 · Cited by 7
Katins, C., Feger, S. S., & Kosch, T. (2023) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3626705.3627785 · Cited by 6
Self-citationLeveraging cognitive computing for advanced behavioral and emotional data insights
Du, C., & Wang, W. (2024) · International Journal of Cognitive Computing in Engineering · doi:10.1016/j.ijcce.2024.12.005 · Cited by 7
Multimodal EEG–EMG and FEM-Based Adaptive Control of Passive Upper-Limb Exoskeletons
Bibbò, L., Laganà, F., Pullano, S. A., & Angiulli, G. (2026) · Sensors · doi:10.3390/s26123924 · Cited by 11
Assessing Eye Tracking for Continuous Central Field Loss Monitoring
Grootjen, J. W., Sipatchin, A., Wahl, S., Machulla, T., Chuang, L. L., & Kosch, T. (2023) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3626705.3627776 · Cited by 6
Self-citationBu, L., Cui, B., Pan, W., Chen, H., Xia, S., & Li, H. (2025) · Universal Access in the Information Society · doi:10.1007/s10209-025-01240-4 · Cited by 6
“The NASA-TLX is widely utilized in cognitive workload research within mul - titasking environments, making it a reliable tool for quan - tifying the cognitive resource consumption of SLVP users across different information transmission methods [57 , 58].”
Pupillometry for Measuring User Response to Movement of an Industrial Robot
Hostettler, D., Bektaş, K., & Mayer, S. (2023) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/3588015.3590123 · Cited by 7
“…interaction (HCI) and HRI literature, multiple objective (e.g., ocular, cerebral, dermal) and subjective metrics (e.g., questionnaires) were used to determine cognitive workload (CWL) and related factors (stress, arousal, frustration) that are mentally demanding for users [Kosch et al. 2023].”
BackgroundMethodInfluentialVitti, M., Cotruvo, A., & Facchini, F. (2025) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-025-02716-z · Cited by 9
“The NASA-TLX questionnaire is by far the most frequently adopted tool, largely due to its simplicity, minimal implementation effort, and adaptability across different task types (Kosch et al., 2023).”
Interpretable Time-Dependent Convolutional Emotion Recognition with Contextual Data Streams
Bethge, D., Patsch, C., Hallgarten, P., & Kosch, T. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585672 · Cited by 6
“cognitive workload [21] or biased expectations”
Self-citationHuman-centered Behavioral and Physiological Security
Alt, F., Hassib, M. E., & Distler, V. (2023) · New Security Paradigms Workshop · doi:10.1145/3633500.3633504 · Cited by 12
Langner, M., Toreini, P., & Maedche, A. (2024) · i-com · doi:10.1515/icom-2023-0035 · Cited by 8
Sun, X., Guo, X., Zhang, Y., & Du, D. (2025) · Ergonomics · doi:10.1080/00140139.2025.2501768 · Cited by 10
“…for driving fatigue. therefore, assessing the cl during driver operation is the primary issue in evaluating hMi. the driving domain mainly uses the sWat, nasa-tlX, driving activity load index (Dali) and workload profile to measure the cl of driving tasks (Rubio et al. 2004; Kosch et al. 2023).”
3DA: Assessing 3D-Printed Electrodes for Measuring Electrodermal Activity
Schmitz, M., Schön, D., Klagemann, H., & Kosch, T. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650938 · Cited by 8
Self-citationMa, Y., & Yao, H. (2025) · Displays · doi:10.1016/j.displa.2024.102891 · Cited by 8
Weight-Induced Consumed Endurance (WICE): A Model to Quantify Shoulder Fatigue with Weighted Objects
Li, T., Velloso, E., Withana, A. I., & Sarsenbayeva, Z. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747745 · Cited by 9
Hierarchical Home Action Understanding with Implicit and Explicit Prior Knowledge
Zhou, Y., Tan, G., & Gou, C. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10445997 · Cited by 5
“We strongly believe our framework has the potential to bridge the gap from existing studies to more real-world applications such as robotics [8, 9, 10, 11], surveillance [12, 13, 14, 15], and human-computer interaction [16, 17, 18, 19, 20].”
BackgroundExploring the Remapping Impact of Spatial Head-hand Relations in Immersive Telesurgery
Luo, T., Ke, Z., Wang, P., Chang, S.-T., Chen, G., Zhang, H., Tan, X., Wang, Q., Han, T., & Tian, F. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714285 · Cited by 6
From Expectations to Attitudes: The Role of Performance Expectancy in New Technology Adoption
Alamsyah, D. P., Othman, N. A., Morika, D., et al. (2025) · International Conference on Wireless and Telematics (ICWT) · doi:10.1109/icwt66752.2025.11181940 · Cited by 4
“Human-computer interaction (HCI) is a part of online learning that supports the process of user adaptation to new technologies [1], [2].”
Liang, X., Long, Y., Xiang, K., Shi, P., Wang, Z., Xiao, K., Kumar, S., Li, X., & Min, R. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3437763 · Cited by 6
“DigitalObject Identifier 10.1109/JSEN.2024.3437763 control [1], [2].”
InfluentialAutomated recognition of mental cognitive workload through nonlinear EEG analysis
Zheng, Z., & Weng, L. (2024) · Web Intelligence · doi:10.3233/web-240141 · Cited by 6
“While the tools are simple to use, they don’t get immediate feedback, objectivity, as well as exact findings [27].”
Designing adaptive feedback systems for managing cognitive load in augmented reality
Sun, J., & Liao, T. (2025) · Design Science · doi:10.1017/dsj.2025.10040 · Cited by 11
“Managing cognitive load is one of the fundamental considerations for designing interactive systems (Kosch et al. 2023; Gkintoni et al. 2025), particularly in immersive environments like augmented reality (AR), where the system complexity and information interference can increase users ’ cognitive…”
Effects of User Interface Design on Cognitive Load in Air Traffic Management
Neuhaus, A. R., Kusmitsch, C. J., Eier, D., & Simmank, F. (2024) · Integrated Communications, Navigation and Surveillance Conference (ICNS) · doi:10.1109/icns60906.2024.10550752 · Cited by 4
“Due to a novel portfolio of sensors for gathering data and AI for effortlessly processing it, operator workload receives even more interest nowadays (compare [2], [17], [18]).”
BackgroundExploring Differential Entropy and Multifractal Cumulants for EEG-based Mental Workload Recognition
Lü, Y. (2024) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2024.0150515 · Cited by 5
“While these approaches are easy to implement, they lack objectivity, real-time feedback, and precise results [5].”
BackgroundHuman-in/on-the-Loop Design for Human Controllability
Cheruvu, R. (2025) · Springer eBooks · doi:10.1007/978-981-97-8440-0_75-1 · Cited by 12
Elgohary, O., & Zhu-Tien (2025) · arXiv · doi:10.48550/arxiv.2509.05898 · Cited by 4
Unified Spatio-Temporal Tracking via Adaptive Embedding and Temporal Context Modeling
Jia, X., Hou, Z., Ma, S., & Pu, L. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116116 · Cited by 3
Pei, H., Zhang, Y., Gong, H., Cao, Y., Ding, M., Bai, Z., & Ding, M. (2026) · Applied Soft Computing · doi:10.1016/j.asoc.2026.115099 · Cited by 5
Cross-task cognitive workload estimation using eye tracking
Yang, L., Wang, L., Xu, W., Wang, B., Ren, H., & Yang, A. (2025) · Signal Image and Video Processing · doi:10.1007/s11760-025-03931-0 · Cited by 4
“Kosch et al. [1] provided a comprehensive survey on cognitive workload measurement in human-computer interaction, particularly highlighting the growing use of eye-tracking data for real-time workload estimation.”
Tudisco, V., Tappia, E., Perotti, S., Meiser, A., & Klumpp, M. (2025) · International Journal of Production Research · doi:10.1080/00207543.2025.2609168 · Cited by 3
“The selection of the assessment approaches aimed to balance the ease of deployment during the experiment, with the robustness and variety of data sources (Kosch et al. 2023).”
Landgrebe, V., Bouquet, E., von der Au, S., & Mayer, S. (2025) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3771882.3771887 · Cited by 5
Rodrigues, L. S., Schmidt, T. T., Nyakatura, J. A., Zachow, S., Israel, J. H., & Kosch, T. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3698137 · Cited by 6
Self-citationA Decision Support System tailored to the Maintenance Activities of Industry 5.0 Operators
Olıverı, L. M., Iacono, N. L., Chıacchıo, F., Facchini, F., & Mossa, G. (2024) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2024.08.118 · Cited by 3
Belardinelli, A., Wang, C., Tanneberg, D., Hasler, S., & Gienger, M. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2025.1605652 · Cited by 6
Cross-domain facial expression recognition: Bi-Directional Fusion of Active and Stable Information
Zhu, Y., Ai, J., Xue, W., Wu, M., Yang, S., Jia, W., & Hu, M. Q. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110357 · Cited by 3
Hossain, M. M., & Mahmud, M. R. (2025) · IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) · doi:10.1109/ismar-adjunct68609.2025.00031 · Cited by 7
Designing a Technique-Oriented Sport Training Game for Motivating a Change in Running Technique
Smith, I., Scheme, E., & Bateman, S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3677064 · Cited by 6
Masked self‐supervised pre‐training model for EEG‐based emotion recognition
Hu, X., Chen, Y., Yan, J., Wu, Y. L., Ding, L., Xu, J., & Cheng, J. T. (2024) · Computational Intelligence · doi:10.1111/coin.12659 · Cited by 4
Mixed Presence in Mixed Reality: Charting the Challenges and Opportunities
Krug, K., Büschel, W., Satkowski, M., Gumhold, S., & Dachselt, R. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791508 · Cited by 2
Haller, O., Meidan, Y., Mimran, D., Elovici, Y., & Shabtai, A. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789151 · Cited by 4
Arakawa, R., Maeda, K., & Yakura, H. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3709012 · Cited by 13
A Human–Computer-Interaction Exploration of Working Memory Using EEG
Kyriaki, K., & Fidas, C. A. (2025) · IEEE Access · doi:10.1109/access.2025.3606802 · Cited by 4
“Kosch et al. examined how CL is conceptualized, presented subjective and objective measurements of CL, including EEG, and suggested a procedure to select the most suitable metrics for CL evaluation in the context of HCI [38].”
Metadata
- Authors
- Kosch, T., Karolus, J., Zagermann, J., Reiterer, H., Schmidt, A., & Woźniak, P. W.
- Affiliations
- Humboldt-Universität zu Berlin (DE)
- German Research Centre for Artificial Intelligence (DE)
- University of Konstanz (DE)
- Ludwig-Maximilians-Universität München (DE)
- Chalmers University of Technology (SE)
- ORCID
- Venue
- ACM Computing Surveys
- Published
- 2023-07-13
- Volume
- 55
- Issue
- 13s
- Pages
- 1-39
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4318618520
- Cited by
- 304
- References
- 259
- Citations by year
- 2026: 120 · 2025: 95 · 2024: 64 · 2023: 25
- FWCI
- 89.72
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Computing Surveys (publishedVersion, cc-by)
- Chalmers Research (Chalmers University of Technology) (submittedVersion)
- Funders
- Deutsche Forschungsgemeinschaft (251654672 – TRR 161 and 414984028 – CRC 1404 FONDA)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- mental workload, workload metrics, NASA-TLX, mental demand, user experience, reproducibility, interactive systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.




