Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 99 of the 99 references it cites were found in a registry or an index; 7 of the 67 sources its claims cite were checked for support.

The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence

Open access
Marucci, Matteo, Di Flumeri, Gianluca, Borghini, Gianluca, Sciaraffa, Nicolina, Scandola, Michele, Pavone, Enea Francesco, Babiloni, Fabio, Betti, Viviana, Aricò, Pietro · 2021 · Scientific Reports

Abstract

Real-world experience is typically multimodal. Evidence indicates that the facilitation in the detection of multisensory stimuli is modulated by the perceptual load, the amount of information involved in the processing of the stimuli. Here, we used a realistic virtual reality environment while concomitantly acquiring Electroencephalography (EEG) and Galvanic Skin Response (GSR) to investigate how multisensory signals impact target detection in two conditions, high and low perceptual load. Different multimodal stimuli (auditory and vibrotactile) were presented, alone or in combination with the visual target. Results showed that only in the high load condition, multisensory stimuli significantly improve performance, compared to visual stimulation alone. Multisensory stimulation also decreases the EEG-based workload. Instead, the perceived workload, according to the "NASA Task Load Index" questionnaire, was reduced only by the trimodal condition (i.e., visual, auditory, tactile). This trimodal stimulation was more effective in enhancing the sense of presence, that is the feeling of being in the virtual environment, compared to the bimodal or unimodal stimulation. Also, we show that in the high load task, the GSR components are higher compared to the low load condition. Finally, the multimodal stimulation (Visual-Audio-Tactile-VAT and Visual-Audio-VA) induced a significant decrease in latency, and a significant increase in the amplitude of the P300 potentials with respect to the unimodal (visual) and visual and tactile bimodal stimulation, suggesting a faster and more effective processing and detection of stimuli if auditory stimulation is included. Overall, these findings provide insights into the relationship between multisensory integration and human behavior and cognition.

Overview

CoChat Summary

## TL;DR

In an immersive VR target-detection task with 18 healthy male volunteers, multisensory cues improved performance only under high perceptual load, while also reducing EEG-based workload and increasing presence. P300 measures and skin conductance supported the view that higher multisensory integration and higher load changed stimulus processing, arousal, and user experience.

## Key findings

  • In the performance LMM, perceptual load had a significant main effect: performance was lower in the high-load than the low-load condition (marginal R² = 0.53; conditional R² = 0.67).
  • There was a significant Condition × Task interaction for performance: in the high-load condition only, bimodal and trimodal stimulation improved performance versus visual-only, with VA p = 0.003, VAT p < 0.001, and VT p = 0.037.
  • Sense of presence increased with multisensory integration and with load: VAT produced higher presence than V (p = 0.007) and VT (p = 0.026), VA was higher than V (p = 0.048), and high load increased presence by β = 0.559 (p = 0.0118).
  • EEG findings showed multisensory benefits in stimulus processing: P300 amplitude increased for VA and VAT versus V and VT (all p-values < 0.001 or < 0.01), while P300 latency increased for VA and VAT versus V and VT; high load also increased latency by β = 0.44.
  • Workload measures diverged by instrument: the EEG-based workload index decreased for VAT and VT versus V (β = 0.333, p = 0.002; β = 0.319, p = 0.006), while NASA-TLX showed a significant reduction only for VAT versus V (p = 0.025).
  • Skin conductance increased under high load for both tonic and phasic components: tonic β = 0.995 (p = 0.004) and phasic β = 1.02 (p < 0.0001).

## Methods

  • The study used a counterbalanced VR experiment with a target-detection driving task under 2 perceptual-load conditions (low vs high) and 4 sensory tasks: V, VA, VT, and VAT.
  • Participants were 18 healthy male volunteers, mean age 26.5 ± 3.2 years, with normal or corrected-to-normal vision.
  • Outcomes included behavioral performance (#HitT and TimeD), subjective workload (NASA-TLX), presence questionnaire scores, EEG-derived workload (WEEG), P300 amplitude/latency, and GSR-based arousal measures; analysis used linear mixed-effects models with subject random effects and post hoc Tukey tests.

## Limitations

  • The paper states that perceptual load may have intrinsically increased cognitive load, and that the design did not allow disentangling cognitive from perceptual load; further studies are needed.
  • The sample included only 18 participants and only male volunteers, which the paper says was done to reduce variability due to putative gender effects.
  • The text makes evident that the study used a highly controlled VR task, so generalization to other settings is uncertain; this is an inference from the experimental design, not a limitation explicitly stated in the paper.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41598-021-84196-8

Cites

99 of 99 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 (A) Experimental setup; (B) the first-person perspective of the low and (C) the high load conditions. The target object (the sphere) is shown in the insert; (D) summary of the experimental procedure with the details for each task. Informed consent was obtained for this image.
Figure 1 (A) Experimental setup; (B) the first-person perspective of the low and (C) the high load conditions. The target object (the sphere) is shown in the insert; (D) summary of the experimental procedure with the details for each task. Informed consent was obtained for this image.CC BY · PubMed Central
Figure 2 Boxplots representing behavioural correlates of the (A) Performance Index for the low load condition (top) and high load condition (bottom) for each sensory task; (B) presence questionnaire scores across the sensory tasks. The questionnaire proposed at the end of each run consisted of three items on a scale from 1 to 7. These scores were averaged to obtain a single value for each condition. The black asterisks indicate significant post-hoc tests while red dots and lines refer to the mean. The central line represents the median, the top and the bottom of the box are the first and third quartiles, and the whiskers are the interquartile range of the lower quartile and of the upper quartile multiplied by 1.5.
Figure 2 Boxplots representing behavioural correlates of the (A) Performance Index for the low load condition (top) and high load condition (bottom) for each sensory task; (B) presence questionnaire scores across the sensory tasks. The questionnaire proposed at the end of each run consisted of three items on a scale from 1 to 7. These scores were averaged to obtain a single value for each condition. The black asterisks indicate significant post-hoc tests while red dots and lines refer to the mean. The central line represents the median, the top and the bottom of the box are the first and third quartiles, and the whiskers are the interquartile range of the lower quartile and of the upper quartile multiplied by 1.5.CC BY · PubMed Central
Figure 3 The figures above (A, B) show respectively the grand average over all the subjects, for each sensory modality and difficulty level, for few representative channels. The figure below shows the significance of the main effect sensory task for the P300 amplitude (C) and latency (D), low and high conditions have been collapsed over both. The black asterisks indicate significant post-hoc tests while red dots and lines refer to the mean. The central line represents the median, the top and the bottom of the box are the first and third quartiles, and the whiskers are the interquartile range of the lower quartile and of the upper quartile multiplied by 1.5.
Figure 3 The figures above (A, B) show respectively the grand average over all the subjects, for each sensory modality and difficulty level, for few representative channels. The figure below shows the significance of the main effect sensory task for the P300 amplitude (C) and latency (D), low and high conditions have been collapsed over both. The black asterisks indicate significant post-hoc tests while red dots and lines refer to the mean. The central line represents the median, the top and the bottom of the box are the first and third quartiles, and the whiskers are the interquartile range of the lower quartile and of the upper quartile multiplied by 1.5.CC BY · PubMed Central
Figure 4 Boxplots representing (A) the EEG-based Workload index and (B) the workload experienced by the subjects, both across the Sensory Tasks. Panel C represents the different scales of the NASA TLX. The black asterisks indicate significant post-hoc tests while red dots and lines refer to the mean. The central line represents the median, the top and the bottom of the box are the first and third quartiles, and the whiskers are the interquartile range of the lower quartile and of the upper quartile multiplied by 1.5. Performance score scale is inverted (from 100 to 0), coherently with the NASA-TLX questionnaire. In fact, in the workload evaluation, a lower performance means a higher perceived workload.
Figure 4 Boxplots representing (A) the EEG-based Workload index and (B) the workload experienced by the subjects, both across the Sensory Tasks. Panel C represents the different scales of the NASA TLX. The black asterisks indicate significant post-hoc tests while red dots and lines refer to the mean. The central line represents the median, the top and the bottom of the box are the first and third quartiles, and the whiskers are the interquartile range of the lower quartile and of the upper quartile multiplied by 1.5. Performance score scale is inverted (from 100 to 0), coherently with the NASA-TLX questionnaire. In fact, in the workload evaluation, a lower performance means a higher perceived workload.CC BY · PubMed Central
Figure 5 Boxplots representing (A) the tonic GSR averaged across sensory tasks for each Load Condition; (B) the phasic GSR averaged across sensory tasks for each load condition. The black asterisks indicate significant post-hoc tests while red dots and lines refer to the mean. The central line represents the median, the top and the bottom of the box are the first and third quartiles, and the whiskers are the interquartile range of the lower quartile and of the upper quartile multiplied by 1.5.
Figure 5 Boxplots representing (A) the tonic GSR averaged across sensory tasks for each Load Condition; (B) the phasic GSR averaged across sensory tasks for each load condition. The black asterisks indicate significant post-hoc tests while red dots and lines refer to the mean. The central line represents the median, the top and the bottom of the box are the first and third quartiles, and the whiskers are the interquartile range of the lower quartile and of the upper quartile multiplied by 1.5.CC BY · PubMed Central

Verification

Claims the paper makes

4 Valid · 2 Not valid · 30 Not checked · 54 Own results and interpretations

37 more claims

Create a free account to see all 90 claims and to verify the 7 references still open.

See all claims — free account
  • It may state two separate claims as one. 30
  • Probably not a substantial claim. 23
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Citations

Cites

Show all 99 references

Research works printed but not linked: 6

  1. Gonzalez-Franco, M., Perez-Marcos, D., Spanlang, B. & Slater, M. in 2010 IEEE virtual reality conference (VR). 111–114 (IEEE).
  2. Cartocci, G. et al. in 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). 1654–1657 (IEEE).
  3. Fox, J. & Weisberg, S. An R Companion to Applied Regression (Third) (Sage, Thousand Oaks CA, 2019).
  4. Lenth, R. & Lenth, M. R. Package ‘lsmeans’. Am. Stat. 34(4), 216–221 (2018).
  5. Osaka, M. et al. in 2012 ICME International Conference on Complex Medical Engineering (CME). 717–721 (IEEE).
  6. Slater, M., Brogni, A. & Steed, A. in Presence 2003: The 6th Annual International Workshop on Presence. (Citeseer).

Cited by

See all 170

Metadata

Authors
Marucci, M., Di Flumeri, G., Borghini, G., Sciaraffa, N., Scandola, M., Pavone, E. F., Babiloni, F., Betti, V., & Aricò, P.
Affiliations
  • Sapienza University of Rome (IT)
  • Fondazione Santa Lucia (IT)
  • University of Verona (IT)
Corresponding author
Marucci, Matteo
Venue
Scientific Reports
Published
2021-03-01
Volume
11
Issue
1
Pages
4831-4831
ISSN
2045-2322 (ISSN-L)
OpenAlex
W3135027244
Cited by
170
References
99
Citations by year
2026: 35 · 2025: 38 · 2024: 39 · 2023: 30 · 2022: 20 · 2021: 8
FWCI
23.44
Type
journal article
Language
en
Open access
Licence
Locations
  • Scientific Reports (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • IRIS Research product catalog (Sapienza University of Rome) (submittedVersion, cc-by)
Funders
  • European Commission (699379)
  • H2020 European Research Council (759651)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
  • Physical Sciences › Computer Science › Human-Computer Interaction › Virtual Reality Applications and Impacts
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
Keywords
multisensory integration, perceptual load, virtual reality, target detection, workload, presence, P300, skin conductance, auditory stimulation, vibrotactile stimulation, multimodal stimuli, NASA-TLX

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence. https://app.cochat.ai/papers/10.1038/s41598-021-84196-8
The paper
Marucci, M., Di Flumeri, G., Borghini, G., Sciaraffa, N., Scandola, M., Pavone, E. F., Babiloni, F., Betti, V., & Aricò, P. (2021). The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence. Scientific Reports. https://doi.org/10.1038/s41598-021-84196-8

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.