Cognitive Heat: Exploring the Usage of Thermal Imaging to Unobtrusively Estimate Cognitive Load
Open accessThe DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
Abstract
Current digital systems are largely blind to users’ cognitive states. Systems that adapt to users’ states show great potential for augmenting cognition and for creating novel user experiences. However, most approaches for sensing cognitive states, and cognitive load specifically, involve obtrusive technologies, such as physiological sensors attached to users’ bodies. This paper present an unobtrusive indicator of the users’ cognitive load based on thermal imaging that is applicable in real-world. We use a commercial thermal camera to monitor a person’s forehead and nose temperature changes to estimate their cognitive load. To assess the effect of different levels of cognitive load on facial temperature we conducted a user study with 12 participants. The study showed that different levels of the Stroop test and the complexity of reading texts affect facial temperature patterns, thereby giving a measure of cognitive load. To validate the feasibility for real-time assessments of cognitive load, we conducted a second study with 24 participants, we analyzed the temporal latency of temperature changes. Our system detected temperature changes with an average latency of 0.7 seconds after users were exposed to a stimulus, outperforming latency in related work that used other thermal imaging techniques. We provide empirical evidence showing how to unobtrusively detect changes in cognitive load in real-time. Our exploration of exposing users to different content types gives rise to thermal-based activity tracking, which facilitates new applications in the field of cognition-aware computing.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Research using thermal imaging found promising results in distinguishing cognitive load levels using standardized cognitive load tests [2].”
From A Survey on Measuring Cognitive Workload in Human-Computer Interaction (2023) · cites this paper as [2]
Citations
Cites
Studies of interference in serial verbal reactions
Stroop, J. R. (1935) · Journal of Experimental Psychology · doi:10.1037/h0054651 · Cited by 18,622
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
The Computer for the 21st Century
Weiser, M. (1991) · Scientific American · doi:10.1038/scientificamerican0991-94 · Cited by 10,017
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382
Detecting Stress During Real-World Driving Tasks Using Physiological Sensors
Healey, J. A., & Picard, R. W. (2005) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2005.848368 · Cited by 2,207
Task-evoked pupillary responses, processing load, and the structure of processing resources.
Beatty, J. (1982) · Psychological Bulletin · doi:10.1037/0033-2909.91.2.276 · Cited by 1,959 · cited 1×
“Different approaches have been introduced to infer cognitive states ranging from using facial expressions [65], eye movements [18] and pupil size [6, 46], skin conductance [20, 32, 56], brain signals [8, 57, 58], Electrodermal activity (EDA) [17, 41, 42], heart and respiration rate [32].”
BackgroundLicklider, J. C. R. (1960) · IRE Transactions on Human Factors in Electronics · doi:10.1109/thfe2.1960.4503259 · Cited by 1,938
AUGMENTING HUMAN INTELLECT: A CONCEPTUAL FRAMEWORK
ENGELBART, D. C. (1962) · doi:10.21236/ad0289565 · Cited by 1,082
Studies of interference in serial verbal reactions.
Stroop, J. R. (1992) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.121.1.15 · Cited by 918
Thermal cameras and applications: A survey
Gade, R., & Moeslund, T. B. (2014) · Machine Vision and Applications · doi:10.1007/s00138-013-0570-5 · Cited by 780
Research works printed but not linked: 4
- Yomna Abdelrahman. 2013. Thermal imaging for interactive surfaces. Master’s thesis. Yomna Abdelrahman. 2013. Thermal imaging for interactive surfaces. Master’s thesis.
- Hosni Sarah M, 2007, International Conference on. IEEE, 220--226
- Kang J, 2006, Proceedings of Army Science Conference
- 10.5555/2062850.2062929 · doi:10.5555/2062850.2062929
Cited by
Items 101–170 of 170
Zhu, H., Ma, L., Zhang, F., Yang, D., Ukai, M., & Hu, S. (2024) · Indoor Air · doi:10.1155/2024/2210991 · Cited by 7
A trifactorial model of detection of deception using thermography
Galvéz-García, G., Fernández‐Gómez, J., Bascour–Sandoval, C., Albayay, J., González-Quiñones, J. J., Moline, A. D., Pérez-Luco, R., Boso, À., & Milán, E. G. (2020) · Psychology Crime and Law · doi:10.1080/1068316x.2020.1815198 · Cited by 7
The MIDAS touch: Thermal dissipation resulting from everyday interactions as a sensing modality
Dar, F., Emenike, H., Yin, Z., Liyanage, M., Sharma, R. R., Zuniga, A., Hoque, M. A., Radeta, M., Nurmi, P., & Flores, H. (2022) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2022.101625 · Cited by 6
Zhong, S., Chwalek, P. C., Perry, N., Ramsay, D. B., Miller, C., Lalanne, D., Alavi, H. S., & Paradiso, J. A. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103510 · Cited by 6
A quick review of infrared thermography studies on children and adolescents’ mental health
Zhao, J., Xu, H. M., Zhang, X., Ma, Y., Zhang, Z., Huo, S., Sun, Y., & Li, Y. (2025) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2025.1515032 · Cited by 6
A Multi-Sensor Approach for Cognitive Load Assessment in Mobile Augmented Reality
Pluisch, M., Gugenheimer, J., Cho, Y., Julier, S., & Kruijff, E. (2025) · Proceedings - International Symposium on Mixed and Augmented Reality, ISMAR · doi:10.1109/ismar67309.2025.00033 · Cited by 6
Gaze assisted voice note taking system
Khan, A. A. (2019) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3341162.3349308 · Cited by 6
Integrating infrared technologies in science learning: An evidence-based reasoning perspective
Pei, B., Xing, W., Zhu, G., Antonyan, K., & Xie, C. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-11538-y · Cited by 6
Exploring the domestication of thermal imaging
Abdelrahman, Y., Woźniak, P. W., Knierim, P., Weber, D., Pfeuffer, K., Henze, N., Schmidt, A., & Alt, F. (2019) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3365610.3365648 · Cited by 6
Self-citationPrivacy and Security in Augmentation Technologies
Khamis, M., & Alt, F. (2021) · Human-computer interaction series · doi:10.1007/978-3-030-30457-7_8 · Cited by 6
Akintola, A., Liyanage, M., Machado, R., Norbisrath, U., Nurmi, P., & Flores, H. (2024) · IEEE Pervasive Computing · doi:10.1109/mprv.2024.3416677 · Cited by 5
Watanabe, K., Gund, G., Santhosh, J., Sakagami, H., Matsuda, Y., Dengel, A. R., & Ishimaru, S. (2025) · IEEE Access · doi:10.1109/access.2025.3578185 · Cited by 5
Designing Thermal Feedback for Notifying Users About Stress
Poguntke, R., Ilk, J., Schmidt, A., & Abdelrahman, Y. (2019) · doi:10.4108/eai.20-5-2019.2283340 · Cited by 5
Self-citationUncovering perceived identification accuracy of in-vehicle biometric sensing
Ali, A. E., Ashby, L., Webb, A. M., Zwitser, R. J., & Cesar, P. S. (2019) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/3349263.3351506 · Cited by 5
MEEC: First Workshop on Momentary Emotion Elicitation and Capture
Ali, A. E., Perusquía-Hernández, M., Denman, P., Abdelrahman, Y., Hassib, M. E., Meschtscherjakov, A., Ferreira, D., & Henze, N. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3375175 · Cited by 5
Self-citationCárdenas, J. J. G., & Tapus, A. (2025) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros60139.2025.11247537 · Cited by 4
Hearing the World: A Pilot Study Design on Spatial Audio for the Visually Impaired
Zhao, X. '. (2024) · International Academic Mindtrek Conference (Mindtrek) · doi:10.1145/3681716.3689442 · Cited by 4
Melo, J., Fernandez, L. B., & Ishimaru, S. (2025) · IEEE Access · doi:10.1109/access.2025.3537156 · Cited by 4
Assessment of Cognitive Workload During Flight Training by Means of Hybrid NIR/LWIR Imaging
Tomasino, D., Chinthakindi, M., Tiberio, A., Patibandla, C., Tritto, M., & Nocco, S. (2023) · IEEE International Workshop on Technologies for Defense and Security (TechDefense) · doi:10.1109/techdefense59795.2023.10380919 · Cited by 4
HeatSight: Wearable Low-power Omni Thermal Sensing
Alharbi, R., Feng, C., Sen, S., Jain, J., Hester, J. D., & Alshurafa, N. I. (2021) · International Symposium on Wearable Computers · doi:10.1145/3460421.3478811 · Cited by 4
Muke, P. Z., Telec, Z., & Trawiński, B. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-21967-2_56 · Cited by 4
Extracting Daytime-Dependent Alertness Patterns from Mobile Game Data
Dingler, T., Singer, K., Henze, N., & Machulla, T. (2020) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/3379503.3403537 · Cited by 3
Self-citationMEEC: Second Workshop on Momentary Emotion Elicitation and Capture
Ali, A. E., Perusquía-Hernández, M., Hassib, M. E., Abdelrahman, Y., & Newn, J. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3441351 · Cited by 3
Self-citationSeptiadi, W. N., Sulistiowati, N. M. D., & Wakhid, A. (2019) · Humanities & Social Sciences Reviews · doi:10.18510/hssr.2019.75149 · Cited by 3
Velloso, E., Dingler, T., Vetere, F., Horman, S., McDougall, H., & Mierzejewska, K. (2018) · Australian Conference on Computer-Human Interaction (OzCHI) · doi:10.1145/3292147.3293451 · Cited by 3
Self-citationAutomated inference of cognitive performance by fusing multimodal information acquired by smartphone
Hamatani, T., Ochiai, K., Inagaki, A., Yamamoto, N., Fukazawa, Y., Kimoto, M., Kiriu, K., Kaminishi, K., Ota, J., Terasawa, Y., et al. (2019) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3341162.3346275 · Cited by 3
Instant Automated Inference of Perceived Mental Stress through Smartphone PPG and Thermal ImagingPreprint
Cho, Y., Julier, S., & Bianchi‐Berthouze, N. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/326157 · Cited by 3
Measuring Mental Workload of Software Developers Based on Nasal Skin Temperature
Nakasai, K., Komeda, S., Tsunoda, M., & Kashima, M. (2024) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2024edl8002 · Cited by 2
CLUES: Cognitive Load Understanding through Experimental Sensing Dataset
Krstevska, A., Goyal, S., Fiorini, L., Bona, F. B. D., Kirilenko, D., Li, M., Trojer, S., Božak, T., Anžur, Z., Luštrek, M., et al. (2026) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3831656 · Cited by 2
Captivates: A Smart Eyeglass Platform for Across-Context Physiological Measurements
Chwalek, P. C., Ramsay, D. B., & Paradiso, J. A. (2023) · GetMobile Mobile Computing and Communications · doi:10.1145/3614214.3614220 · Cited by 2
Dynamic Offset Correction for Smartphone Thermal Cameras Using a Wristband Sensor
Yoshikawa, H., Uchiyama, A., & Teruo Higashino, Y. (2019) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomw.2019.8730732 · Cited by 2
Eye Tracking based Cognitive-Centered User Models
Fidas, C. A. (2019) · IEEE/WIC/ACM International Conference on Web Intelligence · doi:10.1145/3350546.3352563 · Cited by 2
Jonell, P. (2019) · Adaptive Agents and Multi-Agents Systems · Cited by 2
Washington, P. Y. (2023) · medRxiv · doi:10.1101/2023.08.24.23294587 · Cited by 2
Dalilian, F. (2024) · Iowa Research Online (The University of Iowa) · doi:10.25820/etd.007407 · Cited by 2
Nicolaï, A. (2021) · theses.fr (ABES) · doi:10.70675/0afffe4fzf2b0z421bz8fbfz2325b78ee5e1 · Cited by 2
A Bibliometric Analysis of Cognitive Load Sensing Methodologies and Its Applications
Villarreal, R. T., Nordstrom, P. A., & Duffy, V. G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-61060-8_9 · Cited by 1
Kip, A., Erle, T. M., & van Beest, I. (2025) · Psychophysiology · doi:10.1111/psyp.70081 · Cited by 1
Wang, H., Gupta, J., & Martelaro, N. (2025) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/3744333.3747831 · Cited by 1
Learning in the Loop: AI, Feedback, and the Future of Human Thought
Dingler, T. (2026) · Springer eBooks · doi:10.1007/978-981-95-1622-3_9 · Cited by 1
Self-citationThe nose knows: Thermal responses to active psychological stressors
Theroude, P., Paisley, M., Thompson, M., Wheeler, A., Donati, G., & Forrester, G. S. (2026) · PLoS ONE · doi:10.1371/journal.pone.0338108 · Cited by 2
Standard spatial distribution of facial skin temperature: A preliminary study
Nagumo, K., Oiwa, K., & 野澤, 昭. (2026) · Infrared Physics & Technology · doi:10.1016/j.infrared.2026.106437 · Cited by 1
Face Thermal Map of the Mexican Population in the Basal State
Jaramillo-Quintanar, D., Cruz-Albarrán, I. A., Trejo, B. D., Medina, D. A. R., & Morales-Hernandez, L. A. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph192114208 · Cited by 1
From the Laboratory into the Wild: Eyewear in Cognitive-Aware System Studies
Tag, B., & Augereau, O. (2018) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3267305.3274120 · Cited by 1
Abdelrahman, Y. (2020) · Human-computer interaction series · doi:10.1007/978-3-030-15954-2_5 · Cited by 1
Self-citationThe PERFORM Mask: A Psychophysiological sEnsoRs Mask FOr Real-Life Cognitive Monitoring
Menicucci, D., Laurino, M., Marinari, E., Cesari, V., & Gemignani, A. (2020) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-030-49289-2_7 · Cited by 1
Los efectos del estrés social en tiempos de mantener la sana distancia: Termorregulación afectiva
Multidisciplinarias, C. R. D. I., & Medina, D. A. R. (2020) · doi:10.22201/crim.001r.2020.22 · Cited by 1
Ho, B.-J. (2019) · eScholarship (California Digital Library) · Cited by 1
A Virtual Reality Simulation Pipeline for Online Mental Workload ModelingPreprint
Wilson, R. L., Browne, D., Wagstaff, J. M., & McGuire, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.03977 · Cited by 1
CogniDot: Vasoactivity-based Cognitive Load Monitoring with a Miniature On-skin SensorPreprint
Lan, H., Li, Y., Li, S., Yi, X., & Zhang, T. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.19206 · Cited by 1
Learning on the blink: physiological predictors of adaptive learning in an equine model
Evans, L. (2024) · Nottingham Trent University's Institutional Repository (Nottingham Trent Repository) · Cited by 1
Leveraging eye gaze to enhance security mechanisms
Abdrabou Mahmoud, Y. E., Alt, F., Khamis, M., & Deutschland (2023) · Cited by 1
Unobtrusive Identification of Cognitive States for Improved Knowledge Acquisition
Tag, B., & Sugiura, K. (2018) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3267305.3267315
Ubiquitous Technologies for Emotion Recognition
Baños, O., Castro, L. A., & Villalonga, C. (2021) · MDPI eBooks · doi:10.3390/books978-3-0365-1801-5
Predicting Users’ Mental Effort in Drawing Tasks Using Gesture Recognition
Ray, S. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581754.3584104
Multi-modal Data Collection and Preliminary Statistical Analysis for Cognitive Load Assessment
(2024) · Slovenian Conference on Artificial Intelligence · doi:10.70314/is.2024.scai.6961
Roth, Y. G. (2025) · Preprints.org · doi:10.20944/preprints202505.1925.v1
Boltaboyeva, A., Baigarayeva, Z., Zhussupbekov, S., Kalpeyeva, Z. B., & Uskenbayeva, R. (2025) · International Journal of Innovative Research and Scientific Studies · doi:10.53894/ijirss.v8i5.9444
Unveiling faking in job interviews by examining facial thermal cues in deception detection
Hypšová, P., Seitl, M., Kubíček, J., & Mimra, T. (2025) · Scientific Reports · doi:10.1038/s41598-025-29072-5
ヨシカワ, ヒ., & 𠮷川, 寛. (2022) · Institutional Repositories DataBase (IRDB) · doi:10.18910/88150
Cognitive Load Sensing with FLIR Thermal Cameras and the Effects of Calibration Correction
Malmivirta, T. (2020) · Helda - University of Helsinki Institutional Repository
Thermal Camera-Based Cognitive Load Estimation: ANon-Invasive Approach
(2025) · Slovenian Conference on Artificial Intelligence · doi:10.70314/is.2025.skui.3714
Beyond the Interface: Augmenting Meta-Skills for Long-Term Cognitive Enhancement
Ishimaru, S. (2026) · Springer eBooks · doi:10.1007/978-981-95-1622-3_8
Monitoring Nasal Skin Temperature with a Contact Sensor: Insights from a Pilot Study on Stress
Bernasconi, S., Antonello, N., Scandelli, A., Villa, F., Trojaniello, D., & Aliverti, A. (2025) · IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE) · doi:10.1109/metroxraine66377.2025.11340356
A Platform for Physiological and Behavioral Security
Dietz, F., Heubl, P., Haliburton, L., Bothe, D., Hörnemann, J., Sasse, M. A., & Alt, F. (2025) · New Security Paradigms Workshop · doi:10.1145/3774761.3774915
Ye, S., Lin, K., Hou, P., Li, C., & Ai, Z. (2026) · Building and Environment · doi:10.1016/j.buildenv.2026.114651
Cognitive Resilience to Thermal Stress Insafety-Critical Cleanroom Task
Baigarayeva, Z., Boltaboyeva, A., Kabievna, U. R., Кальпеева, Ж., Turmakhan, S., & Turmakhanbet, D. (2026) · Procedia Computer Science · doi:10.1016/j.procs.2026.04.131
Feng, A., Zhao, X., Gao, S., Qian, C. Z., Li, W., & Cheng, A. (2026) · Behavioral Sciences · doi:10.3390/bs16060934
Deb, S., Sharma, H., & Salekin, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.16087
Metadata
- Authors
- Abdelrahman, Y., Velloso, E., Dingler, T., Schmidt, A., & Vetere, F.
- Affiliations
- University of Stuttgart (DE)
- The University of Melbourne (AU)
- Osaka Prefecture University (JP)
- Venue
- Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
- Published
- 2017-09-11
- Volume
- 1
- Issue
- 3
- Pages
- 1-20
- ISSN
- 2474-9567 (ISSN-L)
- OpenAlex
- W2755506150
- Cited by
- 154
- References
- 70
- Citations by year
- 2026: 8 · 2025: 11 · 2024: 14 · 2023: 16 · 2022: 16 · 2021: 21 · 2020: 21 · 2019: 28 · 2018: 15 · 2017: 4
- FWCI
- 22.99
- Type
- journal article
- Language
- en
- Open access
- Locations
- Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies (publishedVersion)
- Open access LMU (Ludwid Maxmilian's Universitat Munchen) (submittedVersion, other-oa)
- Funders
- German Research Foundation within the SimTech Cluster of Excellence (EXC 310/2)
- Amplify project which received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (683008)
- Victorian State Government and Microsoft through their contributions to the Microsoft Research Centre for Social Natural User Interfaces
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- cognitive load, infrared thermography, unobtrusive sensing, forehead temperature, nose temperature, Stroop task, activity monitoring, thermal camera
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.