The DOI matches its Crossref record; 68 of the 72 references it cites were found in a registry or an index; 7 of the 56 sources its claims cite were checked for support.
Study of Multimodal Interfaces and the Improvements on Teleoperation
Open accessAbstract
Research in multimodal interfaces aims to provide immersive solutions and to increase overall human performance. A promising direction is to combine auditory, visual and haptic interaction between the user and the simulated environment. However, no extensive comparison exists to show how combining audiovisuohaptic interfaces would affect human perception and by extent reflected on task performance. Our paper explores this idea and presents a thorough, full-factorial comparison of how all combinations of audio, visual and haptic interfaces affect performance during manipulation. We evaluated how each combination affects the performance in a study (N=25 ) consisting of manipulation tasks with various difficulties. The overall performance was assessed using both subjective measures, by assessing cognitive workload and system usability, and objective measurements, by incorporating time and spatial accuracy-based metrics. The results showed that regardless of task complexity, the combination of stereoscopic-vision with the virtual reality headset increased performance across all measurements by 40%, compared to monocular-vision from a generic display monitor. Besides, using haptic feedback improved outcomes by 10% and auditory feedback accounted for approximately 5% improvement.
Overview
CoChat Summary
## TL;DR
This paper reports a full-factorial study of 8 multimodal teleoperation interface conditions across 25 participants performing pick-and-place manipulation tasks of varying difficulty. The strongest performance gains came from stereoscopic VRHMD vision, with additional benefits from haptic feedback and only limited effects from auditory feedback.
## Key findings
- Across subjective and objective measures, stereoscopic VRHMD vision outperformed the monocular monitor; the paper reports an overall performance increase of around 40% with VRHMD compared to the monitor.
- Adding somatosensory (haptic) feedback improved outcomes by a further 10% overall, but this benefit was significant mainly when paired with stereoscopic vision.
- Auditory feedback had no significant effect on most measures; the paper reports only an approximately 5% improvement in spatial accuracy.
- For task success rate, the stereoscopic conditions (C5–C8) achieved 96.22%, 99.11%, 96.22%, and 97.55%, versus 39.33%, 47.55%, 51.33%, and 48.22% for the monocular conditions (C1–C4).
- For workload and usability, the paper reports strong condition effects: perceived workload F(2.393, 57.437)=70.473, p<0.001, ηp²=0.746 and SUS usability F(3.930, 94.310)=97.064, p<0.001, ηp²=0.802.
## Methods
- Full-factorial repeated-measures experiment with 8 interface conditions combining vision (monitor vs VRHMD), audio (off/on), and haptics (off/on), tested on a pick-and-place task in a virtual environment.
- Sample: 25 participants from the University of Edinburgh, ages 21–44 (mean 26.36, SD 4.829), with 6 females and 19 males.
- Main analyses used RM-ANOVA for normally distributed data, ART + RM-ANOVA for ordinal questionnaire data, Friedman/Wilcoxon for nonparametric time metrics, and a three-sigma rule for Bernoulli success proportions.
## Limitations
- The paper states it did not test how auditory or somatosensory feedback might compensate for overloaded visual information.
- The paper states it did not explicitly and strictly investigate monocular vs stereoscopic visual feedback by themselves.
- Evident from the text: the study uses a static scenario and the authors note that conclusions about vection and motion sickness cannot be fully drawn from it, which may limit generalization to navigation or more dynamic teleoperation tasks.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures

















Verification
Claims the paper makes
6 Valid · 2 Not valid · 33 Not checked · 57 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“constantly providing feedback can be counterproductive both in user preference and time ef ciency, compared to the feedback at the end of a task [23].”
What the sources say
- Supportedcites [23] [23] X. Shang, M. Kallmann, and A. S. Arif, ''Effects of correctness and suggestive feedback on learning with an autonomous virtual trainer,'' in Proc. 24th Int. Conf. Intell. User Interface Companion (IUI) , New York, NY, USA, 2019, pp. 93 94, doi: 10.1145/3308557.3308675. - source
The paper defines SF as continuously providing clues/suggestions during the task and CF as informing users about performance only at the end of a task (Introduction, Sec. 1). Its Results (Sec. 3) report SF took 559 s vs 339 s for CF (65% longer) with no significant learning-outcome difference, and that more participants cited CF as preferable and more effective; the Abstract and Conclusions state CF 'received higher preference and led to shorter task completion time'. This matches the hedged claim that continuous feedback can be less preferable and less time-efficient than end-of-task feedback. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Supplementing vision with auditory information can increase operator's awareness, especially during high visual load [2], [26].”
What the sources say
- Partly supportedcites [2] [2] R. Sigrist, G. Rauter, R. Riener, and P. Wolf, ''Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review,'' Psychonomic Bull. Rev. , vol. 20, no. 1, pp. 21 53, Feb. 2013, doi: 10.3758/s13423012-0333-8. - sourceExperimental — accuracy not yet measured
In “Audiovisual feedback enhances perception,” the review says distributing information across vision and hearing is appropriate in navigation tasks where vision is highly loaded, and reports improved flight or driving performance. It does not establish increased operator awareness; it also reports that audiovisual feedback can be detrimental in tasks with high overall workload. (LLM, conf=0.84)
- Not matchedcites [26] [26] R. D. Shilling and B. Shinn-Cunningham, ''Virtual auditory displays,'' in Handbook Virtual Environments . Boca Raton, FL, USA: CRC Press, 2002, pp. 105 132. <!-- image --> IEEEAccess" -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“These results provide evidence to the untested hypothesis of [2].”
What the sources say
- Supportedcites [2] [2] R. Sigrist, G. Rauter, R. Riener, and P. Wolf, ''Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review,'' Psychonomic Bull. Rev. , vol. 20, no. 1, pp. 21 53, Feb. 2013, doi: 10.3758/s13423012-0333-8. - source
In “Multimodal feedback is promising” (page unavailable), Sigrist et al. explicitly hypothesize that augmented multimodal feedback is more efficient than unimodal feedback for motor learning. They also state that some multimodal combinations had not yet been investigated. The citing authors’ assessment of their own results is not a claim the cited review can verify. (LLM, conf=0.89)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“point-based haptic devices and moreover auditory feedback did not signi cantly contribute to the improvement of task performance.”
What the sources say
- Supportedcites [45] [45] E. Frid, J. Moll, R. Bresin, and E.-L. Salln s Pysander, ''Haptic feedback combined with movement soni cation using a friction sound improves task performance in a virtual throwing task,'' J. Multimodal User Interface , vol. 13, no. 4, pp. 279 290, Dec. 2019, doi: 10.1007/s12193-0180264-4. - source
The Conclusions (p. 7) state the two attributed points directly for a virtual throwing task performed with a point-based haptic device: haptic feedback in general could not be concluded to significantly improve task performance, and 'moreover, the addition of auditory feedback alone could not be shown to significantly improve task performance.' The Discussion (Sect. 6) and Sect. 5.2 corroborate this (hypothesis H2 rejected; quantitative results did not confirm that haptic feedback improved performance), and the Abstract/Experimental design confirm the point-based device and the throwing-task population/setting. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Preliminary ndings indicate that force feedback is only more bene cial than vibrotactile stimulation when presented at a high-resolution [42], [56].”
What the sources say
- Not checked — no full text of the sourcecites [42] [42] C. C. Berger, M. Gonzalez-Franco, E. Ofek, and K. Hinckley, ''The uncanny valley of haptics,'' Sci. Robot. , vol. 3, no. 17, Apr. 2018, Art. no. eaar7010. - source
- Supportedcites [56] [56] B. Weber, M. Sagardia, T. Hulin, and C. Preusche, ''Visual, vibrotactile, and force feedback of collisions in virtual environments: Effects on performance, mental workload and spatial orientation,'' in Virtual Augmented Mixed Reality. Designing Developing Augmented Virtual Environments , R. Shumaker, Ed. Berlin, Germany: Springer, 2013, pp. 241 250. - source
The Discussion (Section 5) states that 'the force feedback system with a light weight robot as input device and high resolution 6 DoF force feedback is superior in terms of precision compared to the vibrotactile and visual feedback systems,' explicitly binding the advantage to a high-resolution force-feedback implementation. The same section adds the converse condition: 'Vibrotactile devices could be a reasonable alternative if high resolution of haptic information is not critical,' which supports the claim's 'only ... at high resolution' framing. Subjective data in the Discussion likewise rate mental workload lower and usability/spatial orientation higher for force feedback, so 'more beneficial' is a fair paraphrase of the paper's precision-plus-workload result; the paper's own superiority statement is scoped to its assembly (peg-in-hole / narrow passage) paradigm, consistent with the citing sentence's 'preliminary findings' hedge. (LLM, conf=0.80)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
60 more claims
Create a free account to see all 98 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 20
- Probably not a substantial claim. 17
- The quote only partly states this claim. 1
Citations
Cites
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003 · cited 3×
“We first measured cognitive workload for each interface condition through the use of the multidimensional assessment tool questionnaire NASA-Task Load Index, simply known as NASA-TLX [62].”
MethodSUS: A 'Quick and Dirty' Usability Scale
(1996) · Usability Evaluation In Industry · doi:10.1201/9781498710411-35 · Cited by 8,989
The use of ranks to avoid the assumption of normality implicit in the analysis of variance
Friedman, M. (1937) · Journal of the American Statistical Association · doi:10.1080/01621459.1937.10503522 · Cited by 7,005 · cited 1×
“For nonparametric continuous data, a Friedman’s test, similar to the RM-ANOVA, was used to test for significance across the eight interface conditions [66], and Wilcoxon signedrank tests for post-hoc analysis for the pairwise comparison across the interface conditions.”
MethodThe Uncanny Valley [From the Field]
Mori, M., MacDorman, K., & Kageki, N. (2012) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2012.2192811 · Cited by 3,542 · cited 1×
“called ‘‘Uncanny Valley’’ – a break in immersion when an artificial being appears too realistic, causing negative responses towards it [49].”
BackgroundThe aligned rank transform for nonparametric factorial analyses using only anova procedures
Wobbrock, J. O., Findlater, L., Gergle, D., & Higgins, J. J. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1978963 · Cited by 2,854 · cited 5×
“For the perceived system usability, an ART was used to allow the use of parametric tests on ordinal data.”
MethodInfluentialMeasuring and defining the experience of immersion in games
Jennett, C., Cox, A. L., Cairns, P., Dhoparee, S., Epps, A., Tijs, T., & Walton, A. (2008) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2008.04.004 · Cited by 2,065 · cited 1×
“This can lead to improved performance when controlling another body in a remote environment [2], [3], and increasing immersion is known to increase the performance [4]–[6].”
BackgroundIs neocortex essentially multisensory?
GHAZANFAR, A., & SCHROEDER, C. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.04.008 · Cited by 1,463 · cited 1×
“Neuroimaging studies support this concept, showing that visual and haptic activation overlaps in the occipital lobe [50]–[53].”
BackgroundAugmented visual, auditory, haptic, and multimodal feedback in motor learning: A review
Sigrist, R., Rauter, G., Riener, R., & Wolf, P. (2013) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0333-8 · Cited by 1,416
Depth of Presence in Virtual Environments
Slater, M., Usoh, M., & Steed, A. (1994) · Presence: Teleoperators and Virtual Environments · doi:10.1162/pres.1994.3.2.130 · Cited by 1,342 · cited 1×
“We implemented both measurements to compensate for the inherent drawbacks of exclusively using questionnaires [60], [61].”
MethodPukelsheim, F. (1994) · The American Statistician · doi:10.2307/2684253 · Cited by 1,242
Research works printed but not linked: 3
- simpson, 2013, Spatial audio display concepts supporting situation awareness for operators of unmanned aerial vehicles, Human Performance Situation Awareness and Automation Current Research and Trends, 2, 61
- richard, 1994, A comparison of haptic, visual and auditive force feedback for deformable virtual objects, Proc Internation Conf Autom Technol (ICAT), 49, 62
- aleotti, 2002, A Multi-modal User Interface for Remote Object Exploration in Teleopera-tion Systems
Cited by
Teleoperation methods and enhancement techniques for mobile robots: A comprehensive survey
Moniruzzaman, M., Rassau, A., Chai, D., & Islam, S. M. S. (2022) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2021.103973 · Cited by 177
A Review of Human–Machine Cooperation in the Robotics Domain
Yang, C., Zhu, Y., & Chen, Y. (2022) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2021.3131684 · Cited by 128
A review on manipulation skill acquisition through teleoperation‐based learning from demonstration
Si, W., Wang, N., & Yang, C. (2021) · Cognitive Computation and Systems · doi:10.1049/ccs2.12005 · Cited by 140
“Extensive comparisons [30] have been done to show that regardless of task complexity, using multimodal interface could improve the performance.”
BackgroundInfluentialTechnologies for Multimodal Interaction in Extended Reality—A Scoping Review
Rakkolainen, I., Farooq, A., Kangas, J. A., Hakulinen, J., Rantala, J., Turunen, M., & Raisamo, R. (2021) · Multimodal Technologies and Interaction · doi:10.3390/mti5120081 · Cited by 136
“Finally, multimodal solutions can increase the level of immersion [228,229] and sometimes also task performance [229,230].”
BackgroundVibrotactile feedback in virtual motor learning: A systematic review
Islam, M. S., & Lim, S. I. (2022) · Applied Ergonomics · doi:10.1016/j.apergo.2022.103694 · Cited by 119
Metadata
- Authors
- Triantafyllidis, E., Mcgreavy, C., Gu, J., & Li, Z.
- Affiliations
- University of Edinburgh (GB)
- Venue
- IEEE Access
- Published
- 2020
- Volume
- 8
- Pages
- 78213-78227
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3019814707
- Cited by
- 105
- References
- 72
- Citations by year
- 2026: 10 · 2025: 17 · 2024: 25 · 2023: 17 · 2022: 16 · 2021: 20
- FWCI
- 6.52
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- Funders
- EPSRC Future AI and Robotics for Space (EP/R026092/1)
- Topics
- Physical Sciences › Engineering › Mechanical Engineering › Teleoperation and Haptic Systems
- Physical Sciences › Computer Science › Human-Computer Interaction › Virtual Reality Applications and Impacts
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Keywords
- multimodal interfaces, teleoperation, haptic feedback, auditory feedback, stereo vision, virtual reality headsets, manipulation tasks, mental workload, system usability, task performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Study of Multimodal Interfaces and the Improvements on Teleoperation. https://app.cochat.ai/papers/10.1109/access.2020.2990080
- The paper
- Triantafyllidis, E., Mcgreavy, C., Gu, J., & Li, Z. (2020). Study of Multimodal Interfaces and the Improvements on Teleoperation. IEEE Access. https://doi.org/10.1109/access.2020.2990080
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.