Comparing the effects of visual-auditory and visual-tactile feedback on user performance: a meta-analysis
Abstract
In a meta-analysis of 43 studies, we examined the effects of multimodal feedback on user performance, comparing visual-auditory and visual-tactile feedback to visual feedback alone. Results indicate that adding an additional modality to visual feedback improves performance overall. Both visual-auditory feedback and visual-tactile feedback provided advantages in reducing reaction times and improving performance scores, but were not effective in reducing error rates. Effects are moderated by task type, workload, and number of tasks. Visual-auditory feedback is most effective when a single task is being performed (g = .87), and under normal workload conditions (g = .71). Visual-tactile feedback is more effective when multiple tasks are begin performed (g = .77) and workload conditions are high (g = .84). Both types of multimodal feedback are effective for target acquisition tasks; but vary in effectiveness for other task types. Implications for practice and research are discussed.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
People have different cognitive resources for information processing, though not all can be used simultaneously without interference.
“For instance, Burke et al. (2006) has stated that people have different cognitive resources for information processing, even though not all can be used simultaneously without interference.”
From Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review (2013) · cites this paper as Burke et al., 2006
The use of both auditory and visual channels increases workload because the two modalities are cognitively linked.
“In that meta-analysis, it is suggested that the use of both the auditory and the visual channel increases workload because the two modalities are cognitively linked (Burke et al., 2006).”
From Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review (2013) · cites this paper as Burke et al., 2006
Visuotactile displays were reported to be most effective in various tasks with high workload.
“Visuotactile displays were reported to be most effective in various tasks with high workload (Burke et al., 2006).”
From Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review (2013) · cites this paper as Burke et al., 2006
Adding more modalities is correlated to improved performance in manipulation scenarios.
“This is in line with existing literature, that adding more modalities is correlated to improved performance in manipulation scenarios [46].”
From Study of Multimodal Interfaces and the Improvements on Teleoperation (2020) · cites this paper as [46]
Cites
Multiple resources and performance prediction
Wickens, C. D. (2002) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220210123806 · Cited by 2,621 · cited 3× · see citing works
“This is compelling support for the theoretical arguments presented in Multiple Resource Theory [45], since the positive effects of adding the tactile modality are more pronounced when workload is high.”
BackgroundResultInfluentialMayer, R. E., & Moreno, R. (1998) · Journal of Educational Psychology · doi:10.1037/0022-0663.90.2.312 · Cited by 1,315 · see citing works
Vibrotactile in-vehicle navigation system
Van Erp, J. B. F., & Van Veen, H. A. H. C. (2004) · Transportation Research Part F: Traffic Psychology and Behaviour · doi:10.1016/j.trf.2004.09.003 · Cited by 337 · see citing works
Liu, Y.-C. (2001) · Ergonomics · doi:10.1080/00140130010011369 · Cited by 246 · see citing works
Evaluating Adaptive Multisensory Displays for Target Localization in a Flight Task
Tannen, R. S., Nelson, W. T., Bolia, R. S., Warm, J. S., & Dember, W. N. (2004) · The International Journal of Aviation Psychology · doi:10.1207/s15327108ijap1403_5 · Cited by 31 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review
Sigrist, R., Rauter, G., Riener, R., & Wolf, P. (2013) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0333-8 · Cited by 1,416 · see citing works
Cooper, N., Milella, F., Pinto, C., Cant, I., White, M., & Meyer, G. (2018) · PLOS ONE · doi:10.1371/journal.pone.0191846 · Cited by 201 · see citing works
Study of Multimodal Interfaces and the Improvements on Teleoperation
Triantafyllidis, E., Mcgreavy, C., Gu, J., & Li, Z. (2020) · IEEE Access · doi:10.1109/access.2020.2990080 · Cited by 105 · see citing works
“This is in line with existing literature, that adding more modalities is correlated to improved performance in manipulation scenarios [46].”
BackgroundResultEffects of Multimodal Error Feedback on Human Performance in Steering Tasks
Sun, M., Ren, X., & Cao, X. (2010) · Journal of Information Processing · doi:10.2197/ipsjjip.18.284 · Cited by 39 · see citing works
Impact of Tactile and Visual Feedback on Breathing Rhythm and User Experience in VR Exergaming
Greinacher, R., Kojic, T., Meier, L., Parameshappa, R. G., Moller, S., & Voigt-Antons, J.-N. (2020) · 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex48832.2020.9123141 · Cited by 20 · see citing works
- Authors
- Burke, J. L., Prewett, M. S., Gray, A. A., Yang, L., Stilson, F. R. B., Coovert, M. D., Elliot, L. R., & Redden, E.
- Affiliations
- University of South Florida (US)
- DEVCOM Army Research Laboratory (US)
- Venue
- Proceedings of the 8th international conference on Multimodal interfaces
- Publisher
- ACM
- Published
- 2006-11-02
- Pages
- 108-117
- OpenAlex
- W2091354492
- Cited by
- 221
- References
- 5
- Citations by year
- 2026: 4 · 2025: 14 · 2024: 14 · 2023: 13 · 2022: 12 · 2021: 20 · 2020: 14 · 2019: 21 · 2018: 13 · 2017: 7 · 2016: 8 · 2015: 8 · 2014: 5 · 2013: 17 · 2012: 11
- FWCI
- 7.24
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (publishedVersion)
- Funders
- Army Research Laboratory
- Topics
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- multimodal feedback, visual feedback, user performance, reaction time, error rate, workload, task type, target acquisition, human-computer interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
