Using fNIRS brain sensing in realistic HCI settings: experiments and guidelines
Abstract
Because functional near-infrared spectroscopy (fNIRS) eases many of the restrictions of other brain sensors, it has potential to open up new possibilities for HCI research. From our experience using fNIRS technology for HCI, we identify several considerations and provide guidelines for using fNIRS in realistic HCI laboratory settings. We empirically examine whether typical human behavior (e.g. head and facial movement) or computer interaction (e.g. keyboard and mouse usage) interfere with brain measurement using fNIRS. Based on the results of our study, we establish which physical behaviors inherent in computer usage interfere with accurate fNIRS sensing of cognitive state information, which can be corrected in data analysis, and which are acceptable. With these findings, we hope to facilitate further adoption of fNIRS brain sensing technology in HCI research.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
fNIRS has been demonstrated by HCI researchers to measure user cognitive workload in adaptive systems and is resilient to subject movement such as playing piano.
“Brain sensing using functional near-infrared spectroscopy (fNIRS) has recently been demonstrated by HCI researchers to measure user cognitive workload in adaptive systems [4,44,1] and is resilient to subject movement as might be seen while playing a piano [43].”
From Learn Piano with BACh: An Adaptive Learning Interface that Adjusts Task Difficulty Based on Brain State (2016) · cites this paper as [43]
The fNIRS signal is resilient to respiration, heartbeat, eye movement, minor head motion, and mouse and keyboard clicks.
“The fNIRS signal has been found to be resilient to respiration, heartbeat, eye movement, minor head motion, and mouse and keyboard clicks [43].”
From Learn Piano with BACh: An Adaptive Learning Interface that Adjusts Task Difficulty Based on Brain State (2016) · cites this paper as [43]
Cites
Non-invasive optical spectroscopy and imaging of human brain function
Villringer, A., & Chance, B. (1997) · Trends in Neurosciences · doi:10.1016/s0166-2236(97)01132-6 · Cited by 1,621 · see citing works
Anterior prefrontal cortex: insights into function from anatomy and neuroimaging
Ramnani, N., & Owen, A. M. (2004) · Nature Reviews Neuroscience · doi:10.1038/nrn1343 · Cited by 1,170 · see citing works
Sitaram, R., Zhang, H., Guan, C., Thulasidas, M., Hoshi, Y., Ishikawa, A., Shimizu, K., & Birbaumer, N. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.11.005 · Cited by 590 · see citing works
Functional near-infrared spectroscopy
Bunce, S. C., İzzetoğlu, M., İzzetoğlu, K., Onaral, B., & Pourrezaei, K. (2006) · IEEE Engineering in Medicine and Biology Magazine · doi:10.1109/memb.2006.1657788 · Cited by 319 · see citing works
Creativity meets neuroscience: Experimental tasks for the neuroscientific study of creative thinking
Fink, A., Benedek, M., Grabner, R. H., Staudt, B., & Neubauer, A. C. (2007) · Methods · doi:10.1016/j.ymeth.2006.12.001 · Cited by 260 · see citing works
Functional near-infrared spectroscopy
Gajera, J., & Campos, A. (2020) · Radiopaedia.org · doi:10.53347/rid-82780 · Cited by 239 · see citing works
Functional Optical Brain Imaging Using Near-Infrared During Cognitive Tasks
İzzetoğlu, K., Bunce, S. C., Onaral, B., Pourrezaei, K., & Chance, B. (2004) · International Journal of Human-Computer Interaction · doi:10.1207/s15327590ijhc1702_6 · Cited by 231 · see citing works
Ehlis, A.-C., Bähne, C. G., Jacob, C. P., Herrmann, M. J., & Fallgatter, A. J. (2008) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2007.11.011 · Cited by 209 · see citing works
Folley, B. S., & Park, S. (2005) · Schizophrenia Research · doi:10.1016/j.schres.2005.06.016 · Cited by 201 · see citing works
Feasibility and pragmatics of classifying working memory load with an electroencephalograph
Grimes, D. B., Tan, D. S., Hudson, S. E., Shenoy, P., & Rao, R. P. N. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357187 · Cited by 196 · see citing works
Motion Artifact Cancellation in NIR Spectroscopy Using Wiener Filtering
İzzetoğlu, M., Devaraj, A., Bunce, S. C., & Onaral, B. (2005) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2005.845243 · Cited by 173 · see citing works
(2009) · Cited by 166 · see citing works
Using a low-cost electroencephalograph for task classification in HCI research
Lee, J. C., & Tan, D. S. (2006) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/1166253.1166268 · Cited by 164 · see citing works
Hemodynamics for Brain-Computer Interfaces
Matthews, F., Pearlmutter, B. A., Ward, T. E., Soraghan, C. J., & Markham, C. (2008) · IEEE Signal Processing Magazine · doi:10.1109/msp.2008.4408445 · Cited by 154 · see citing works
Millán, J. D. R., Frédéric Renkens, B., Josep Mouriño, C., & Gerstner, W. (2004) · Artificial Intelligence · doi:10.1016/j.artint.2004.05.008 · Cited by 142 · see citing works
Herrmann, M. J., Huter, T. J., Plichta, M. M., Ehlis, A., Alpers, G. W., Mühlberger, A., & Fallgatter, A. J. (2007) · Human Brain Mapping · doi:10.1002/hbm.20368 · Cited by 107 · see citing works
Brain measurement for usability testing and adaptive interfaces
Hirshfield, L., Solovey, E., Girouard, A., Kebinger, J., Jacob, R. J. K., Sassaroli, A., & Fantini, S. (2009) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1519035 · Cited by 106 · see citing works
Self-citationDistinguishing Difficulty Levels with Non-invasive Brain Activity Measurements
Girouard, A., Solovey, E., Hirshfield, L., Chauncey, K., Sassaroli, A., Fantini, S., & Jacob, R. J. K. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-03655-2_50 · Cited by 67 · see citing works
Self-citationHirshfield, L., Chauncey, K., Gulotta, R., Girouard, A., Solovey, E., Jacob, R. J. K., Sassaroli, A., & Fantini, S. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02812-0_28 · Cited by 62 · see citing works
Self-citationCoyle, S. M., Ward, T. E., & Markham, C. (2005) · doi:10.1109/iembs.2004.1404260 · Cited by 53 · see citing works
Towards a Physiological Model of User Interruptability
Chen, D., Hart, J., & Vertegaal, R. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-74800-7_39 · Cited by 40 · see citing works
A New Brain Imaging Device Based on fNIRS
Chénier, F., & Sawan, M. (2007) · IEEE Biomedical Circuits and Systems Conference (BioCAS) · doi:10.1109/biocas.2007.4463294 · Cited by 14 · see citing works
Near-Infrared Spectroscopy for Studying Higher Cognition
Hoshi, Y. (2008) · On thinking · doi:10.1007/978-3-540-68044-4_6 · Cited by 12 · see citing works
An fNIR based BMI for letter construction using continuous control
Mappus, R. L., Venkatesh, G. R., Shastry, C., et al. (2009) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1520340.1520536 · Cited by 12 · see citing works
Frontal Lobe Activity during the Performance of Spatial Tasks: fNIRS Study
Morioka, S., Yamada, M., & Komori, T. (2008) · Journal of Physical Therapy Science · doi:10.1589/jpts.20.135 · Cited by 7 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Yuksel, B. F., Oleson, K. B., Harrison, L., Peck, E. M., Afergan, D., Chang, R., & Jacob, R. J. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858388 · Cited by 137 · see citing works
Self-citationBrainput: enhancing interactive systems with streaming fnirs brain input
Solovey, E., Schermerhorn, P., Scheutz, M., Sassaroli, A., Fantini, S., & Jacob, R. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2208372 · Cited by 86 · see citing works
Self-citationAlhabash, S., Almutairi, N., Lou, C., & Kim, W. (2019) · Media Psychology · doi:10.1080/15213269.2017.1416296 · Cited by 80 · see citing works
Toward Measurement of Situation Awareness in Autonomous Vehicles
Sirkin, D., Martelaro, N., Johns, M., & Ju, W. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025822 · Cited by 61 · see citing works
Afergan, D., Shibata, T., Hincks, S. W., Peck, E. M., Yuksel, B. F., Chang, R., & Jacob, R. J. K. (2014) · Proceedings of the 27th annual ACM symposium on User interface software and technology · doi:10.1145/2642918.2647414 · Cited by 19 · see citing works
Self-citation
- Authors
- Solovey, E. T., Girouard, A., Chauncey, K., Hirshfield, L. M., Sassaroli, A., Zheng, F., Fantini, S., & Jacob, R. J. K.
- Affiliations
- Tufts University (US)
- ORCID
- Venue
- Proceedings of the 22nd annual ACM symposium on User interface software and technology
- Publisher
- ACM
- Published
- 2009-10-04
- Pages
- 157-166
- OpenAlex
- W2141088043
- Cited by
- 115
- References
- 25
- Citations by year
- 2026: 1 · 2025: 5 · 2024: 2 · 2023: 7 · 2022: 4 · 2021: 3 · 2020: 9 · 2019: 9 · 2018: 7 · 2017: 7 · 2016: 8 · 2015: 12 · 2014: 12 · 2013: 8 · 2012: 4
- FWCI
- 17.30
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Symposium on User Interface Software and Technology (UIST) (publishedVersion)
- Funders
- National Science Foundation
- Microsoft Research
- Air Force Research Laboratory
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Optical Imaging and Spectroscopy Techniques
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
- Keywords
- functional near-infrared spectroscopy, human-computer interaction, head movement, facial motion, artifact removal
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
