Measuring Cognitive Load and Insight: A Methodology Exemplified in a Virtual Reality Learning Context
The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Recent improvements of Virtual Reality (VR) technology have enabled researchers to investigate the benefits VR may provide for various domains such as health, entertainment, training, and education. A significant proportion of VR system evaluations rely on perception-based measures such as user pre-and post-questionnaires and interviews. While these self-reports provide valuable insights into users' perceptions of VR environments, recent developments in digital sensors and data collection techniques afford researchers access to measures of physiological response. This work explores the merits of physiological measures in the evaluation of emotional responses in virtual environments (ERVE). We include and place at the center of our ERVE methodology emotional response data by way of electrodermal activity and heart-rate detection which are analyzed in conjunction with event-driven data to derive further measures. In this paper, we present our ERVE methodology together with a case study within the context of VR-based learning in which we derive measures of cognitive load and moments of insight. We discuss our methodology, and its potential for use in many other application and research domains to provide more in-depth and objective analyses of experiences within VR.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003
SUS: A 'Quick and Dirty' Usability Scale
(1996) · Usability Evaluation In Industry · doi:10.1201/9781498710411-35 · Cited by 8,989
Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness
Kennedy, R. S., Lane, N. E., Berbaum, K. S., & Lilienthal, M. G. (1993) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap0303_3 · Cited by 5,457
DEAP: A Database for Emotion Analysis ;Using Physiological Signals
Koelstra, S., Muhl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., & Patras, I. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.15 · Cited by 5,093
The Experience of Presence: Factor Analytic Insights
Schubert, T., Friedmann, F., & Regenbrecht, H. (2001) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474601300343603 · Cited by 2,303
Self-citationConstruction and Evaluation of a User Experience Questionnaire
Laugwitz, B., Held, T., & Schrepp, M. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-89350-9_6 · Cited by 2,114
Merchant, Z., Goetz, E. T., Cifuentes, L., Keeney-Kennicutt, W., & Davis, T. J. (2014) · Computers & Education · doi:10.1016/j.compedu.2013.07.033 · Cited by 2,109
Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
Calvo, R. A., & D'Mello, S. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.1 · Cited by 1,893 · cited 1×
“Measuring emotional activity by monitoring physiological states is not a novel idea [10,35,37,53].”
BackgroundFrom presence to consciousness through virtual reality.
Sanchez-Vives, M. V., & Slater, M. (2005) · Nature reviews. Neuroscience · doi:10.1038/nrn1651 · Cited by 1,831 · cited 1×
“Sanchez-Vives et al. argues that VR experiences are relevant for the neuroscience space by presenting multiple studies which utilize physiological measurements [56].”
BackgroundVirtual Reality: How Much Immersion Is Enough?
Bowman, D. A., & McMahan, R. P. (2007) · Computer · doi:10.1109/mc.2007.257 · Cited by 1,375
Review: Electrodermal Responses: What Happens in the Brain
Critchley, H. D. (2002) · The Neuroscientist · doi:10.1177/107385840200800209 · Cited by 997 · cited 1×
“This measures the sympathetic nervous system (SNS) and is a sensitive index of sympathetic arousal which is integrated with emotional and cognitive states [14].”
BackgroundA Wearable Sensor for Unobtrusive, Long-Term Assessment of Electrodermal Activity
Poh, M.-Z., Swenson, N. C., & Picard, R. W. (2010) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2009.2038487 · Cited by 693 · cited 1×
“Measuring emotional activity by monitoring physiological states is not a novel idea [10,35,37,53].”
BackgroundPresence: concept, determinants, and measurement:
IJsselsteijn, W. A., de Ridder, H., Freeman, J., & Avons, S. E. (2000) · SPIE Proceedings · doi:10.1117/12.387188 · Cited by 529 · cited 1×
“The research community has started to adopt measures that are more objective and are using them in conjunction with existing subjective measures [31].”
BackgroundAutomatic identification of artifacts in electrodermal activity data
Taylor, S., Jaques, N., Chen, W., Fedor, S., Sano, A., & Picard, R. (2015) · 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2015.7318762 · Cited by 279 · cited 3×
“67% for artifact detection with those labels [69].”
BackgroundMethodMonitoring of mental workload levels during an everyday life office-work scenario
Cinaz, B., Arnrich, B., La Marca, R., & Tröster, G. (2013) · Personal and Ubiquitous Computing · doi:10.1007/s00779-011-0466-1 · Cited by 235 · cited 1×
“The 20% rule for artifact detection is commonly used in the literature [13].”
MethodApplication of virtual reality technology in biology education
Shim, K.-C., Park, J.-S., Kim, H.-S., Kim, J.-H., Park, Y.-C., & Ryu, H.-I. (2003) · Journal of Biological Education · doi:10.1080/00219266.2003.9655854 · Cited by 152 · cited 1×
“The research community has been able to apply VR within multiple topic spaces such as physics [78], chemistry [40,49], biology [59], and geometry [65].”
BackgroundVirtual reality exposure in anxiety disorders: Impact on psychophysiological reactivity
Diemer, J., Mühlberger, A., Pauli, P., & Zwanzger, P. (2014) · The World Journal of Biological Psychiatry · doi:10.3109/15622975.2014.892632 · Cited by 146 · cited 1×
“This measure has prior application in VR environments in the context of anxiety and therapy treatment [17, 20, 75, 76], emotional replication studies [21], behavioural studies [24], and presence [16,45,46,51].”
BackgroundTemporal dynamics in the relation between presence and fear in virtual reality
Peperkorn, H. M., Diemer, J., & Mühlberger, A. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.02.028 · Cited by 82 · cited 2×
“This measure has prior application in VR environments in the context of anxiety and therapy treatment [17, 20, 75, 76], emotional replication studies [21], behavioural studies [24], and presence [16,45,46,51].”
BackgroundFelnhofer, A., Kothgassner, O. D., Hetterle, T., Beutl, L., Hlavacs, H., & Kryspin-Exner, I. (2014) · Cyberpsychology, Behavior, and Social Networking · doi:10.1089/cyber.2013.0472 · Cited by 70 · cited 2×
“This measure has prior application in VR environments in the context of anxiety and therapy treatment [17, 20, 75, 76], emotional replication studies [21], behavioural studies [24], and presence [16,45,46,51].”
Background
Cited by
A Survey on Measuring Cognitive Workload in Human-Computer Interaction
Kosch, T., Karolus, J., Zagermann, J., Reiterer, H., Schmidt, A., & Woźniak, P. W. (2023) · ACM Computing Surveys · doi:10.1145/3582272 · Cited by 304
“…[1, 10, 54, 68, 99, 140, 153, 184, 219, 228] Smooth Pursuits and Ocular Movements [133] Cardiovascular Heart Rate [4, 10, 25, 27, 40, 41, 49, 50, 76, 78, 82, 85, 87, 99, 106, 146, 152, 160, 169, 184, 185, 194, 201, 202, 214, 216, 217, 224, 225, 227, 240, 244, 264, 264, 266, 268] Respiration…”
A systematic review of immersive virtual reality for industrial skills training
Radhakrishnan, U., Koumaditis, K., & Chinello, F. (2021) · Behaviour & Information Technology · doi:10.1080/0144929x.2021.1954693 · Cited by 200
“Collins et al. (2019) used skin conductance, temperature, pulse and heart rate data to understand the emotional responses felt by users when they gained insights into the solution while performing an abstract task of manipulating a hypercube, whereas Š kola et al. ( Š kola, Tinková, and Liarokapis…”
BackgroundMethodInfluentialIntroducing Mental Workload Assessment for the Design of Virtual Reality Training Scenarios
Luong, T., Argelaguet, F., Martin, N., & Lecuyer, A. (2020) · 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr46266.2020.00089 · Cited by 37
Metadata
- Authors
- Collins, J., Regenbrecht, H., Langlotz, T., Said Can, Y., Ersoy, C., & Butson, R.
- Venue
- 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 351-362
- OpenAlex
- W2999233838
- Cited by
- 52
- References
- 19
- FWCI
- 4.73
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.