CoChat
Sign Up
Cited by 34Usability and User Interface DesignOptical Imaging and Spectroscopy TechniquesHuman-Automation Interaction and Safety

Using fNIRS in Usability Testing: Understanding the Effect of Web Form Layout on Mental Workload

Open access
Lukanov, Kristiyan, Maior, Horia A., Wilson, Max L.·2016·Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
DOI 10.1145/2858036.2858236
Open this paper in CoChat — free accountOpen PDF

Abstract

Amongst the many tasks in our lives, we encounter web forms on a regular basis, whether they are mundane like registering for a website, or complex and important like tax returns. There are many aspects of Usability, but one concern for user interfaces is to reduce mental workload and error rates. Whilst most assessment of mental workload is subjective and retrospective reporting by users, we examine the potential of functional Near Infrared Spectroscopy (fNIRS) as a tool for objectively and concurrently measuring mental workload during usability testing. We use this technology to evaluate the design of three different form layouts for a car insurance claim process, and show that a form divided into subforms increases mental workload, contrary to our expectations. We conclude that fNIRS is highly suitable for objectively examining mental workload during usability testing, and will therefore be able to provide more detailed insight than summative retrospective assessments. Further, for the fNIRS community, we show that the technology can easily move beyond typical psychology tasks, and be used for more natural study tasks.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/2858036.2858236

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Gamma power indicates high mental activity at the somatosensory cortex.

    “In contrast, gamma power indicates high mental activity at the somatosensory cortex [141].”

    From A Survey on Measuring Cognitive Workload in Human-Computer Interaction (2023) · cites this paper as [141]

    The Measuring Mental Workload and Objective Measures sections describe fNIRS measures of blood oxygenation in the prefrontal cortex, not gamma power in the somatosensory cortex. Searches for the claim’s phrasing and for gamma, EEG, and somatosensory activity found no passage addressing the claimed relationship. The paper’s use of a different measure does not itself contradict the claim. (LLM, conf=0.77)

    Experimental — accuracy not yet measured

  • Beta power is widely used as a metric for mental workload, including alertness, arousal, frustration, engagement, and workload states.

    “Similarly, beta power is widely used as a metric for mental workload, including alertness, arousal, frustration, engagement, and workload states [141].”

    From A Survey on Measuring Cognitive Workload in Human-Computer Interaction (2023) · cites this paper as [141]

    The Measuring Mental Workload and Objective Measures sections discuss psychophysiological measures and fNIRS haemoglobin measurements, but do not address beta power. Searches for beta power and EEG spectral power found no passage linking beta power to alertness, arousal, frustration, engagement, or workload states. The available passages have no page numbers. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 34

  1. Daniel Afergan, Evan M Peck, Erin T Solovey, Andrew Jenkins, Samuel W Hincks, Eli T Brown, Remco Chang, and Robert JK Jacob. 2014. Dynamic difficulty using brain metrics of workload. In Proc. SIGCHI. ACM, 3797-3806.
  2. Hasan Ayaz. 2010. Functional Near Infrared Spectroscopy based Brain Computer Interface. PhD Thesis, Drexel University, Philadelphia, PA (2010).
  3. Hasan Ayaz, Meltem Izzetoglu, Scott Bunce, Terry Heiman-Patterson, and Banu Onaral. 2007. Detecting cognitive activity related hemodynamic signal for brain computer interface using functional near infrared spectroscopy. In Neural Engineering, 2007. CNE'07. 3rd International IEEE/EMBS Conference on. IEEE, 342-345.
  4. Richard W Backs and Kimberle A Seljos. 1994. Metabolic and cardiorespiratory measures of mental effort: the effects of level of difficulty in a working memory task. International Journal of psychophysiology 16, 1 (1994), 57-68.
  5. Michela Balconi, Elisabetta Grippa, and Maria Elide Vanutelli. 2015. What hemodynamic (fNIRS), electrophysiological (EEG) and autonomic integrated measures can tell us about emotional processing. Brain and cognition 95 (2015), 67-76.
  6. Jackson Beatty. 1982. Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychological bulletin 91, 2 (1982), 276.
  7. Nigel Bevan. 2001. International standards for HCI and usability. International journal of human-computer studies 55, 4 (2001), 533-552.
  8. Nigel Bevan and Ian Curson. 1997. Methods for measuring usability. In Human-Computer Interaction INTERACTâ Ȃ Ź97. Springer, 672-673.
  9. Margaret M Bradley and Peter J Lang. 1994. Measuring emotion: the self-assessment manikin and the semantic differential. Journal of behavior therapy and experimental psychiatry 25, 1 (1994), 49-59.
  10. Todd S Braver, Jonathan D Cohen, Leigh E Nystrom, John Jonides, Edward E Smith, and Douglas C Noll. 1997. A parametric study of prefrontal cortex involvement in human working memory. Neuroimage 5, 1 (1997), 49-62.
  11. Brad Cain. 2007. A review of the mental workload literature. Technical Report. DTIC Document.
  12. Ann Chadwick-Dias, Michelle McNulty, and Tom Tullis. 2003. Web usability and age: how design changes can improve performance. In ACM SIGCAPH Computers and the Physically Handicapped. ACM, 30-37.
  13. C Collet, E Salvia, and C Petit-Boulanger. 2014. Measuring workload with electrodermal activity during common braking actions. Ergonomics 57, 6 (2014), 886-896.
  14. M Cope and David T Delpy. 1988. System for long-term measurement of cerebral blood and tissue oxygenation on newborn infants by near infra-red transillumination. Medical and Biological Engineering and Computing 26, 3 (1988), 289-294.
  15. Xu Cui, Signe Bray, and Allan L Reiss. 2010. Functional near infrared spectroscopy (NIRS) signal improvement based on negative correlation between oxygenated and deoxygenated hemoglobin dynamics. Neuroimage 49, 4 (2010), 3039-3046.
  16. Leanne M Hirshfield, Erin Treacy Solovey, Audrey Girouard, James Kebinger, Robert JK Jacob, Angelo Sassaroli, and Sergio Fantini. 2009. Brain measurement for usability testing and adaptive interfaces: an example of uncovering syntactic workload with functional near infrared spectroscopy. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2185-2194.
  17. Caroline Jarrett and Gerry Gaffney. 2009. Forms that work: Designing Web forms for usability. Morgan Kaufmann.
  18. Lorna Lines, Oluchi Ikechi, K Hone, and Tony Elliman. 2006. Online form design for older adults: Introducing web-automated personalisation. In Proceedings of HCI, the Web and the Older Population, workshop at HCI 2006.
  19. H Maior, Matthew Pike, Sarah Sharples, and Max L Wilson. 2015. Examining the reliability of using fNIRS in realistic hci settings for spatial and verbal tasks. Proceedings of CHI 15 (2015), 3807-3816.
  20. Horia A Maior, Matthew Pike, Max L Wilson, and Sarah Sharples. 2014. Continuous detection of workload overload: An fnirs approach. Contemporary Ergonomics and Human Factors 2014: Proceedings of the international conference on Ergonomics & Human Factors 2014, Southampton, UK, 7-10 April 2014 (2014), 450.
  21. Valerie Mendoza and David G Novick. 2005. Usability over time. In Proceedings of the 23rd annual international conference on Design of communication: documenting & designing for pervasive information. ACM, 151-158.
  22. Erika Molteni, Michele Butti, Anna M Bianchi, and Gianluigi Reni. 2008. Activation of the prefrontal cortex during a visual n-back working memory task with varying memory load: a near infrared spectroscopy study. In Engineering in Medicine and Biology Society, 2008. EMBS 2008. 30th Annual International Conference of the IEEE. IEEE, 4024-4027.
  23. Jakob Nielsen and Jonathan Levy. 1994. Measuring usability: preference vs. performance. Commun. ACM 37, 4 (1994), 66-75.
  24. Jakob Nielsen and Rolf Molich. 1990. Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 249-256.
  25. Evan M M Peck, Beste F Yuksel, Alvitta Ottley, Robert JK Jacob, and Remco Chang. 2013. Using fNIRS brain sensing to evaluate information visualization interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 473-482.
  26. Reinhard Pekrun, Thomas Goetz, Lia M Daniels, Robert H Stupnisky, and Raymond P Perry. 2010. Boredom in achievement settings: Exploring control-value antecedents and performance outcomes of a neglected emotion. Journal of Educational Psychology 102, 3 (2010), 531.
  27. Gary B Reid and Thomas E Nygren. 1988. The subjective workload assessment technique: A scaling procedure for measuring mental workload. Advances in psychology 52 (1988), 185-218.
  28. Sergio Sayago, José-María Guijarro, and Josep Blat. 2012. Selective attention in web forms: an exploratory case study with older people. Behaviour & Information Technology 31, 2 (2012), 171-184.
  29. Yu Shi, Natalie Ruiz, Ronnie Taib, Eric Choi, and Fang Chen. 2007. Galvanic skin response (GSR) as an index of cognitive load. In CHI'07 extended abstracts on Human factors in computing systems. ACM, 2651-2656.
  30. Ben Shneiderman. 1992. Designing the user interface: strategies for effective human-computer interaction. Vol. 3. Addison-Wesley Reading, MA. 60-63 pages.
  31. Erin Treacy Solovey, Audrey Girouard, Krysta Chauncey, Leanne M Hirshfield, Angelo Sassaroli, Feng Zheng, Sergio Fantini, and Robert JK Jacob. 2009. Using fNIRS brain sensing in realistic HCI settings: experiments and guidelines. In Proceedings of the 22nd annual ACM symposium on User interface software and technology. ACM, 157-166.
  32. John Stemberger, Robert S Allison, and Thomas Schnell. 2010. Thermal imaging as a way to classify cognitive workload. In Computer and Robot Vision (CRV), 2010 Canadian Conference On. IEEE, 231-238.
  33. John R Wilson and Sarah Sharples. 2015. Evaluation of human work. CRC Press. 521-522 pages.
  34. Erik WÃd'stlund, Torsten Norlander, and Trevor Archer. 2008. The effect of page layout on mental workload: A dual-task experiment. Computers in Human Behavior 24, 3 (2008), 1229 -1245. DOI: http://dx.doi.org/10.1016/j.chb.2007.05.001 Instructional Support for Enhancing Students' Information Problem Solving Ability. · DOI

Cited by

Sorted by citations

Authors
Lukanov, K., Maior, H. A., & Wilson, M. L.
Affiliations
University of Nottingham (GB)
ORCID
Venue
Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2016-05-07
Pages
4011-4016
DOI
10.1145/2858036.2858236
OpenAlex
W2397450108
Cited by
34
References
4
Citations by year
2026: 1 · 2025: 2 · 2024: 3 · 2023: 5 · 2022: 4 · 2021: 1 · 2020: 5 · 2019: 6 · 2018: 4 · 2017: 3
FWCI
2.96
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion)
  • Lincoln Repository (University of Lincoln) (acceptedVersion)
  • Nottingham ePrints (University of Nottingham) (submittedVersion)
  • Lincoln Repository (University of Lincoln) (submittedVersion)
  • Figshare (submittedVersion)
  • Repository@Nottingham (University of Nottingham) (submittedVersion, other-oa)
Funders
Engineering and Physical Sciences Research Council (EP/G037574/1 and EP/M000877/1)
Topics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Optical Imaging and Spectroscopy Techniques
  • Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
Keywords
functional near-infrared spectroscopy, mental workload, usability testing, web forms, automobile insurance claims, simultaneous measurement, user interface design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review