Study of Multimodal Interfaces and the Improvements on Teleoperation
Open accessThe DOI matches its Crossref record; 68 of the 72 references it cites were found in a registry or an index; 7 of the 56 sources its claims cite were checked for support.
Abstract
Research in multimodal interfaces aims to provide immersive solutions and to increase overall human performance. A promising direction is to combine auditory, visual and haptic interaction between the user and the simulated environment. However, no extensive comparison exists to show how combining audiovisuohaptic interfaces would affect human perception and by extent reflected on task performance. Our paper explores this idea and presents a thorough, full-factorial comparison of how all combinations of audio, visual and haptic interfaces affect performance during manipulation. We evaluated how each combination affects the performance in a study (N=25 ) consisting of manipulation tasks with various difficulties. The overall performance was assessed using both subjective measures, by assessing cognitive workload and system usability, and objective measurements, by incorporating time and spatial accuracy-based metrics. The results showed that regardless of task complexity, the combination of stereoscopic-vision with the virtual reality headset increased performance across all measurements by 40%, compared to monocular-vision from a generic display monitor. Besides, using haptic feedback improved outcomes by 10% and auditory feedback accounted for approximately 5% improvement.
Overview
CoChat Summary
## TL;DR
This paper reports a full-factorial study of 8 multimodal teleoperation interface conditions across 25 participants performing pick-and-place manipulation tasks of varying difficulty. The strongest performance gains came from stereoscopic VRHMD vision, with additional benefits from haptic feedback and only limited effects from auditory feedback.
## Key findings
- Across subjective and objective measures, stereoscopic VRHMD vision outperformed the monocular monitor; the paper reports an overall performance increase of around 40% with VRHMD compared to the monitor.
- Adding somatosensory (haptic) feedback improved outcomes by a further 10% overall, but this benefit was significant mainly when paired with stereoscopic vision.
- Auditory feedback had no significant effect on most measures; the paper reports only an approximately 5% improvement in spatial accuracy.
- For task success rate, the stereoscopic conditions (C5–C8) achieved 96.22%, 99.11%, 96.22%, and 97.55%, versus 39.33%, 47.55%, 51.33%, and 48.22% for the monocular conditions (C1–C4).
- For workload and usability, the paper reports strong condition effects: perceived workload F(2.393, 57.437)=70.473, p<0.001, ηp²=0.746 and SUS usability F(3.930, 94.310)=97.064, p<0.001, ηp²=0.802.
## Methods
- Full-factorial repeated-measures experiment with 8 interface conditions combining vision (monitor vs VRHMD), audio (off/on), and haptics (off/on), tested on a pick-and-place task in a virtual environment.
- Sample: 25 participants from the University of Edinburgh, ages 21–44 (mean 26.36, SD 4.829), with 6 females and 19 males.
- Main analyses used RM-ANOVA for normally distributed data, ART + RM-ANOVA for ordinal questionnaire data, Friedman/Wilcoxon for nonparametric time metrics, and a three-sigma rule for Bernoulli success proportions.
## Limitations
- The paper states it did not test how auditory or somatosensory feedback might compensate for overloaded visual information.
- The paper states it did not explicitly and strictly investigate monocular vs stereoscopic visual feedback by themselves.
- Evident from the text: the study uses a static scenario and the authors note that conclusions about vection and motion sickness cannot be fully drawn from it, which may limit generalization to navigation or more dynamic teleoperation tasks.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures

















Verification
Claims the paper makes
6 Valid · 2 Not valid · 33 Not checked · 57 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“constantly providing feedback can be counterproductive both in user preference and time ef ciency, compared to the feedback at the end of a task [23].”
What the sources say
- Supportedcites [23] [23] X. Shang, M. Kallmann, and A. S. Arif, ''Effects of correctness and suggestive feedback on learning with an autonomous virtual trainer,'' in Proc. 24th Int. Conf. Intell. User Interface Companion (IUI) , New York, NY, USA, 2019, pp. 93 94, doi: 10.1145/3308557.3308675. - source
The paper defines SF as continuously providing clues/suggestions during the task and CF as informing users about performance only at the end of a task (Introduction, Sec. 1). Its Results (Sec. 3) report SF took 559 s vs 339 s for CF (65% longer) with no significant learning-outcome difference, and that more participants cited CF as preferable and more effective; the Abstract and Conclusions state CF 'received higher preference and led to shorter task completion time'. This matches the hedged claim that continuous feedback can be less preferable and less time-efficient than end-of-task feedback. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Supplementing vision with auditory information can increase operator's awareness, especially during high visual load [2], [26].”
What the sources say
- Partly supportedcites [2] [2] R. Sigrist, G. Rauter, R. Riener, and P. Wolf, ''Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review,'' Psychonomic Bull. Rev. , vol. 20, no. 1, pp. 21 53, Feb. 2013, doi: 10.3758/s13423012-0333-8. - sourceExperimental — accuracy not yet measured
In “Audiovisual feedback enhances perception,” the review says distributing information across vision and hearing is appropriate in navigation tasks where vision is highly loaded, and reports improved flight or driving performance. It does not establish increased operator awareness; it also reports that audiovisual feedback can be detrimental in tasks with high overall workload. (LLM, conf=0.84)
- Not matchedcites [26] [26] R. D. Shilling and B. Shinn-Cunningham, ''Virtual auditory displays,'' in Handbook Virtual Environments . Boca Raton, FL, USA: CRC Press, 2002, pp. 105 132. <!-- image --> IEEEAccess" -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“These results provide evidence to the untested hypothesis of [2].”
What the sources say
- Supportedcites [2] [2] R. Sigrist, G. Rauter, R. Riener, and P. Wolf, ''Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review,'' Psychonomic Bull. Rev. , vol. 20, no. 1, pp. 21 53, Feb. 2013, doi: 10.3758/s13423012-0333-8. - source
In “Multimodal feedback is promising” (page unavailable), Sigrist et al. explicitly hypothesize that augmented multimodal feedback is more efficient than unimodal feedback for motor learning. They also state that some multimodal combinations had not yet been investigated. The citing authors’ assessment of their own results is not a claim the cited review can verify. (LLM, conf=0.89)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“point-based haptic devices and moreover auditory feedback did not signi cantly contribute to the improvement of task performance.”
What the sources say
- Supportedcites [45] [45] E. Frid, J. Moll, R. Bresin, and E.-L. Salln s Pysander, ''Haptic feedback combined with movement soni cation using a friction sound improves task performance in a virtual throwing task,'' J. Multimodal User Interface , vol. 13, no. 4, pp. 279 290, Dec. 2019, doi: 10.1007/s12193-0180264-4. - source
The Conclusions (p. 7) state the two attributed points directly for a virtual throwing task performed with a point-based haptic device: haptic feedback in general could not be concluded to significantly improve task performance, and 'moreover, the addition of auditory feedback alone could not be shown to significantly improve task performance.' The Discussion (Sect. 6) and Sect. 5.2 corroborate this (hypothesis H2 rejected; quantitative results did not confirm that haptic feedback improved performance), and the Abstract/Experimental design confirm the point-based device and the throwing-task population/setting. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Preliminary ndings indicate that force feedback is only more bene cial than vibrotactile stimulation when presented at a high-resolution [42], [56].”
What the sources say
- Not checked — no full text of the sourcecites [42] [42] C. C. Berger, M. Gonzalez-Franco, E. Ofek, and K. Hinckley, ''The uncanny valley of haptics,'' Sci. Robot. , vol. 3, no. 17, Apr. 2018, Art. no. eaar7010. - source
- Supportedcites [56] [56] B. Weber, M. Sagardia, T. Hulin, and C. Preusche, ''Visual, vibrotactile, and force feedback of collisions in virtual environments: Effects on performance, mental workload and spatial orientation,'' in Virtual Augmented Mixed Reality. Designing Developing Augmented Virtual Environments , R. Shumaker, Ed. Berlin, Germany: Springer, 2013, pp. 241 250. - source
The Discussion (Section 5) states that 'the force feedback system with a light weight robot as input device and high resolution 6 DoF force feedback is superior in terms of precision compared to the vibrotactile and visual feedback systems,' explicitly binding the advantage to a high-resolution force-feedback implementation. The same section adds the converse condition: 'Vibrotactile devices could be a reasonable alternative if high resolution of haptic information is not critical,' which supports the claim's 'only ... at high resolution' framing. Subjective data in the Discussion likewise rate mental workload lower and usability/spatial orientation higher for force feedback, so 'more beneficial' is a fair paraphrase of the paper's precision-plus-workload result; the paper's own superiority statement is scoped to its assembly (peg-in-hole / narrow passage) paradigm, consistent with the citing sentence's 'preliminary findings' hedge. (LLM, conf=0.80)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
60 more claims
Create a free account to see all 98 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 20
- Probably not a substantial claim. 17
- The quote only partly states this claim. 1
Citations
Cites
Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research
(1988) · Advances in Psychology · doi:10.1016/s0166-4115(08)62386-9 · Cited by 15,003 · cited 3×
“We first measured cognitive workload for each interface condition through the use of the multidimensional assessment tool questionnaire NASA-Task Load Index, simply known as NASA-TLX [62].”
MethodSUS: A 'Quick and Dirty' Usability Scale
(1996) · Usability Evaluation In Industry · doi:10.1201/9781498710411-35 · Cited by 8,989
The use of ranks to avoid the assumption of normality implicit in the analysis of variance
Friedman, M. (1937) · Journal of the American Statistical Association · doi:10.1080/01621459.1937.10503522 · Cited by 7,005 · cited 1×
“For nonparametric continuous data, a Friedman’s test, similar to the RM-ANOVA, was used to test for significance across the eight interface conditions [66], and Wilcoxon signedrank tests for post-hoc analysis for the pairwise comparison across the interface conditions.”
MethodThe Uncanny Valley [From the Field]
Mori, M., MacDorman, K., & Kageki, N. (2012) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2012.2192811 · Cited by 3,542 · cited 1×
“called ‘‘Uncanny Valley’’ – a break in immersion when an artificial being appears too realistic, causing negative responses towards it [49].”
BackgroundThe aligned rank transform for nonparametric factorial analyses using only anova procedures
Wobbrock, J. O., Findlater, L., Gergle, D., & Higgins, J. J. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1978963 · Cited by 2,854 · cited 5×
“For the perceived system usability, an ART was used to allow the use of parametric tests on ordinal data.”
MethodInfluentialMeasuring and defining the experience of immersion in games
Jennett, C., Cox, A. L., Cairns, P., Dhoparee, S., Epps, A., Tijs, T., & Walton, A. (2008) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2008.04.004 · Cited by 2,065 · cited 1×
“This can lead to improved performance when controlling another body in a remote environment [2], [3], and increasing immersion is known to increase the performance [4]–[6].”
BackgroundIs neocortex essentially multisensory?
GHAZANFAR, A., & SCHROEDER, C. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.04.008 · Cited by 1,463 · cited 1×
“Neuroimaging studies support this concept, showing that visual and haptic activation overlaps in the occipital lobe [50]–[53].”
BackgroundAugmented visual, auditory, haptic, and multimodal feedback in motor learning: A review
Sigrist, R., Rauter, G., Riener, R., & Wolf, P. (2013) · Psychonomic Bulletin & Review · doi:10.3758/s13423-012-0333-8 · Cited by 1,416
Depth of Presence in Virtual Environments
Slater, M., Usoh, M., & Steed, A. (1994) · Presence: Teleoperators and Virtual Environments · doi:10.1162/pres.1994.3.2.130 · Cited by 1,342 · cited 1×
“We implemented both measurements to compensate for the inherent drawbacks of exclusively using questionnaires [60], [61].”
MethodPukelsheim, F. (1994) · The American Statistician · doi:10.2307/2684253 · Cited by 1,242
Pukelsheim, F. (1994) · The American Statistician · doi:10.1080/00031305.1994.10476030 · Cited by 1,211 · cited 1×
“empirical rule [67].”
Vision and Touch: An Experimentally Created Conflict between the Two Senses
Rock, I., & Victor, J. (1964) · Science · doi:10.1126/science.143.3606.594 · Cited by 702 · cited 2×
“Our results support existing research that vision is the dominant sense [14], [15], outperforming all other senses [13].”
BackgroundResultHuman Performance Issues and User Interface Design for Teleoperated Robots
Chen, J. Y. C., Haas, E. C., & Barnes, M. J. (2007) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/tsmcc.2007.905819 · Cited by 646 · cited 2×
“This study focuses on audiovisuohaptic interfaces, since vision, hearing and touch are the highest developed and contributing the most towards embodiment [3], [11], [13] among all human senses.”
BackgroundVisuo-haptic object-related activation in the ventral visual pathway
Amedi, A., Malach, R., Hendler, T., Peled, S., & Zohary, E. (2001) · Nature Neuroscience · doi:10.1038/85201 · Cited by 630
Presence: concept, determinants, and measurement:
IJsselsteijn, W. A., de Ridder, H., Freeman, J., & Avons, S. E. (2000) · SPIE Proceedings · doi:10.1117/12.387188 · Cited by 529 · cited 1×
“We implemented both measurements to compensate for the inherent drawbacks of exclusively using questionnaires [60], [61].”
MethodSpatial Updating of Self-Position and Orientation During Real, Imagined, and Virtual Locomotion
Klatzky, R. L., Loomis, J. M., Beall, A. C., Chance, S. S., & Golledge, R. G. (1998) · Psychological Science · doi:10.1111/1467-9280.00058 · Cited by 482 · cited 1×
“Our results support existing research that vision is the dominant sense [14], [15], outperforming all other senses [13].”
ResultHaptic study of three-dimensional objects activates extrastriate visual areas
James, T. W., Humphrey, G. K., Gati, J. S., Servos, P., Menon, R. S., & Goodale, M. A. (2002) · Neuropsychologia · doi:10.1016/s0028-3932(02)00017-9 · Cited by 418
Light, C. M., Chappell, P. H., & Kyberd, P. J. (2002) · Archives of Physical Medicine and Rehabilitation · doi:10.1053/apmr.2002.32737 · Cited by 417 · cited 6×
“Grabbing techniques employed included Precision Grasping via Palmar Pinch [47].”
MethodInfluentialVection and visually induced motion sickness: how are they related?
Keshavarz, B., Riecke, B. E., Hettinger, L. J., & Campos, J. L. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00472 · Cited by 342 · cited 1×
“This is known as vection [18], which is a common complaint among VRHMD users in nonstatic situations.”
BackgroundJudging Perceived and Traversed Distance in Virtual Environments
Witmer, B. G., & Kline, P. B. (1998) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474698565640 · Cited by 335 · cited 2×
“This supports current literature [8], [20], [21].”
BackgroundGu, X., Zhang, Y., Sun, W., Bian, Y., Zhou, D., & Kristensson, P. O. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858487 · Cited by 327 · cited 1×
“Further research in kinesthetic force feedback has shown some advantages over lower-cost approaches [39], [40], particularly due”
BackgroundTactile Display of Vibratory Information in Teleoperation and Virtual Environments
Kontarinis, D. A., & Howe, R. D. (1995) · Presence: Teleoperators and Virtual Environments · doi:10.1162/pres.1995.4.4.387 · Cited by 251 · cited 1×
“Vibratory feedback, however, can be more beneficial than force feedback in direct manipulation tasks such as ours [58].”
BackgroundStereoscopic 3D displays and human performance: A comprehensive review
McIntire, J. P., Havig, P. R., & Geiselman, E. E. (2014) · Displays · doi:10.1016/j.displa.2013.10.004 · Cited by 225 · cited 1×
“The dominance of vision in the sensory system is well supported [14]–[16], contributing to around 70% of overall human perception [13].”
BackgroundBurke, J. L., Prewett, M. S., Gray, A. A., Yang, L., Stilson, F. R. B., Coovert, M. D., Elliot, L. R., & Redden, E. (2006) · Proceedings of the 8th international conference on Multimodal interfaces · doi:10.1145/1180995.1181017 · Cited by 221 · cited 2×
“This is in line with existing literature, that adding more modalities is correlated to improved performance in manipulation scenarios [46].”
BackgroundResultWolverine: A wearable haptic interface for grasping in virtual reality
Choi, I., Hawkes, E. W., Christensen, D. L., Ploch, C. J., & Follmer, S. (2016) · 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759169 · Cited by 216 · cited 1×
“to the ability to constrain the grasp motion of users hands, based on the virtual object they are holding [41].”
BackgroundHaptic feedback: a brief history from telepresence to virtual reality
Stone, R. J. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44589-7_1 · Cited by 213 · cited 2×
“a robot as an avatar to complete a task [1].”
BackgroundA Study on Immersion and Presence of a Portable Hand Haptic System for Immersive Virtual Reality
Kim, M., Jeon, C., & Kim, J. (2017) · Sensors · doi:10.3390/s17051141 · Cited by 175 · cited 2×
“In a more previous study investigating vibrotactile approach, vibration intensity applied to users was proportional to the size of the virtual object being manipulated [55].”
BackgroundMethod"Where am i?" acquiring situation awareness using a remote robot platform
Yanco, H. A., & Drury, J. (2005) · 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583) · doi:10.1109/icsmc.2004.1400762 · Cited by 163
Feeling with the mind's eye: contribution of visual cortex to tactile perception
Sathian, K., & Zangaladze, A. (2002) · Behavioural Brain Research · doi:10.1016/s0166-4328(02)00141-9 · Cited by 158
EL Cine del Futuro: The Cinema of the Future
Heilig, M. L. (1992) · Presence: Teleoperators and Virtual Environments · doi:10.1162/pres.1992.1.3.279 · Cited by 154
Design of a wearable tactile display
Gemperle, F., Ota, N., & Siewiorek, D. (2002) · Proceedings Fifth International Symposium on Wearable Computers · doi:10.1109/iswc.2001.962082 · Cited by 142 · cited 1×
“In particular, manipulation can be improved by adding tactile feedback [1], [35], [36] that can result in better performance [37].”
BackgroundAir jet driven force feedback in virtual reality
Suzuki, Y., & Kobayashi, M. (2005) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2005.1 · Cited by 135 · cited 1×
“Air jet-driven approaches exist for force feedback, while these show significant effectiveness, they nonetheless require large space and pose a substantially higher cost compared to vibration approaches [57].”
BackgroundSensory Substitution for Force Feedback in Teleoperation
Massimino, M. J., & Sheridan, T. B. (1993) · Presence: Teleoperators and Virtual Environments · doi:10.1162/pres.1993.2.4.344 · Cited by 130 · cited 1×
“Overall vibrotactile stimulation is shown to be an effective substitute for force feedback according to another study [59].”
ResultLathan, C. E., & Tracey, M. (2002) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474602760204282 · Cited by 123 · cited 3×
“On one hand, an audiovisuohaptic interface did not significantly increase performance during a teleoperated navigation task [9], but operator’s spatial ability and subjective performance did increase compared to using fewer interfaces.”
BackgroundCholewiak, R. W., & Collins, A. A. (2000) · Perception & Psychophysics · doi:10.3758/bf03212124 · Cited by 116 · cited 1×
“Communicating spatial alerts via somatosensory stimulation can signal warnings without overloading visual pathways [33], [34].”
BackgroundBerger, C. C., Gonzalez-Franco, M., Ofek, E., & Hinckley, K. (2018) · Science Robotics · doi:10.1126/scirobotics.aar7010 · Cited by 109 · cited 8×
“Their study [42] used handheld controllers to”
BackgroundMethodInfluentialMatching and Reaching Depth Judgments with Real and Augmented Reality Targets
Swan, J. E., Singh, G., & Ellis, S. R. (2015) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2459895 · Cited by 108 · cited 2×
“This supports current literature [8], [20], [21].”
BackgroundSecoli, R., Milot, M.-H., Rosati, G., & Reinkensmeyer, D. J. (2011) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/1743-0003-8-21 · Cited by 107 · cited 1×
“Though extra information, such as alarms and alerts, can be superimposed on a visual display, presenting them via an audio interface is a better approach that can decrease distraction [29].”
BackgroundMultimodal virtual reality: Input‐output devices, system integration, and human factors
Burdea, G., Richard, P., & Coiffet, P. (1996) · International Journal of Human-Computer Interaction · doi:10.1080/10447319609526138 · Cited by 102 · cited 2×
“By adding multiple modalities, it was found that the workload of the visual cortex can be reduced, the awareness may be increased and thus the task performance can be improved [9], [10].”
BackgroundMuJoCo HAPTIX: A virtual reality system for hand manipulation
Kumar, V., & Todorov, E. (2015) · 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2015.7363441 · Cited by 102 · cited 2×
“We integrated three basic types of three-dimensional object shapes to not only introduce the inherent different complexities that come with such objects, but also allowing us to assess different grasping techniques [47], [48].”
BackgroundMurray, A. M., Klatzky, R. L., & Khosla, P. K. (2003) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474603321640923 · Cited by 99 · cited 1×
“Outputting vibrations proportional to the force applied by the robot also leads to improved performance [44], and therefore we used a similar approach.”
MethodCapstanCrunch: A Haptic VR Controller with User-supplied Force Feedback
Sinclair, M., Ofek, E., Gonzalez-Franco, M., & Holz, C. (2019) · Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3332165.3347891 · Cited by 95 · cited 1×
“Further research in kinesthetic force feedback has shown some advantages over lower-cost approaches [39], [40], particularly due”
BackgroundCan You See Me Now? How Field of View Affects Collaboration in Robotic Telepresence
Johnson, S., Rae, I., Mutlu, B., & Takayama, L. A. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702526 · Cited by 90 · cited 1×
“Providing a larger field of view can also result in increased performance and environmental awareness [22], [24], [25], but can decrease usability and increase perceived difficulty and workload demand [25].”
BackgroundDistance Estimation in Virtual Environments
Lampton, D. R., McDonald, D. P., Singer, M., & Bliss, J. P. (1995) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193129503902006 · Cited by 82 · cited 3×
“This supports current literature [8], [20], [21].”
BackgroundTelexistence cockpit for humanoid robot control
Tachi, S., Komoriya, K., Sawada, K., Nishiyama, T., Itoko, T., Kobayashi, M., & Inoue, K. (2003) · Advanced Robotics · doi:10.1163/156855303764018468 · Cited by 79 · cited 1×
“humanoid during manipulation tasks [28].”
Image parameters for driving with indirect viewing systems
Van Erp, J. B., & Padmos, P. (2003) · Ergonomics · doi:10.1080/0014013032000121624 · Cited by 78 · cited 1×
“High-resolution displays were chosen to limit distance overestimation and a degraded longitudinal control [54].”
(2014) · Handbook of Virtual Environments · doi:10.1201/b17360-11 · Cited by 74
Realistic force feedback for virtual reality based diagnostic surgery simulators
Vuskovic, V., Kauer, M., Szekely, G., & Reidy, M. (2002) · Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065) · doi:10.1109/robot.2000.844824 · Cited by 71 · cited 1×
“complex and sophisticated tactile approaches of force feedback have been developed to reflect realistic reaction forces of deformable objects such as soft tissue [38].”
BackgroundObstacle avoidance during human walking: learning rate and cross‐modal transfer
Erni, T., & Dietz, V. (2001) · The Journal of Physiology · doi:10.1111/j.1469-7793.2001.00303.x · Cited by 66 · cited 1×
“Further studies in human walking also show that controlling auditory pitch may influence object clearance, with results indicating that participants indeed benefited from such sound sonification [31], [32].”
Immersive 3-D teleoperation of a search and rescue robot using a head-mounted display
Martins, H., & Ventura, R. (2009) · 2009 IEEE Conference on Emerging Technologies & Factory Automation · doi:10.1109/etfa.2009.5347014 · Cited by 63 · cited 2×
“As such, increased depth information leads to reduced collisions with the surrounding environment and better performance during highly dexterous manipulation tasks [7], [22].”
BackgroundChoi, I., & Follmer, S. (2016) · Proceedings of the 29th Annual Symposium on User Interface Software and Technology · doi:10.1145/2984751.2985725 · Cited by 62
(2000) · PubMed · Cited by 60
Improving teleoperation: reducing mental rotations and translations
DeJong, B. P., Colgate, J. E., & Peshkin, M. A. (2004) · IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004 · doi:10.1109/robot.2004.1308838 · Cited by 59
Rosenberg, L. B. (1993) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vrais.1993.380802 · Cited by 58
Yem, V., Vu, K., Kon, Y., & Kajimoto, H. (2018) · 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr.2018.8446403 · Cited by 47 · cited 1×
“In particular, manipulation can be improved by adding tactile feedback [1], [35], [36] that can result in better performance [37].”
Background(2010) · Encyclopedia of Research Design · doi:10.4135/9781412961288.n168 · Cited by 46
Weber, B., Sagardia, M., Hulin, T., & Preusche, C. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-39405-8_28 · Cited by 27
Evaluation of Visual and Auditory Feedback in Virtual Obstacle Walking
Wellner, M., Schaufelberger, A., Zitzewitz, J. V., & Riener, R. (2008) · Presence: Teleoperators and Virtual Environments · doi:10.1162/pres.17.5.512 · Cited by 23 · cited 1×
“Further studies in human walking also show that controlling auditory pitch may influence object clearance, with results indicating that participants indeed benefited from such sound sonification [31], [32].”
Frid, E., Moll, J., Bresin, R., & Sallnäs Pysander, E.-L. (2019) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-018-0264-4 · Cited by 19
The effect of stereoscopic and wide field of view conditions on teleoperator performance
Scribner, D. R., & Gombash, J. W. (1998) · Cited by 17
Smyth, C. C. (2000) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120004403624 · Cited by 11 · cited 1×
“Providing a larger field of view can also result in increased performance and environmental awareness [22], [24], [25], but can decrease usability and increase perceived difficulty and workload demand [25].”
BackgroundBrooks, J., Lodge, R., & White, D. (2017) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213601863 · Cited by 11 · cited 2×
“specific study of a target detection task in Unmanned Aerial Vehicles (UAVs), there were no significant differences in performance between the two [17], with the VRHMD even causing motion sickness potentially attributed to the illusion of self-motion of the vehicle.”
BackgroundYem, V., Vu, K., Kon, Y., & Kajimoto, H. (2018) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr.2018.8446516 · Cited by 10
Multimodal Interaction Modeling
Hale, K. S., & Stanney, K. M. (2014) · CRC Press eBooks · doi:10.1201/b17360-26 · Cited by 8 · cited 2×
“This study focuses on audiovisuohaptic interfaces, since vision, hearing and touch being the highest developed and contributing the most to embodiment [3], [11], [13] among all human senses.”
Considerations for Presence in Teleoperation
McGlynn, S. A., & Rogers, W. A. (2017) · Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/3029798.3038369 · Cited by 5
Effects of correctness and suggestive feedback on learning with an autonomous virtual trainer
Shang, X., Kallmann, M., & Arif, A. S. (2019) · Companion Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3308557.3308675 · Cited by 5 · cited 1×
“One study showed that constantly providing feedback can be counterproductive both in user preference and time efficiency, compared to the feedback at the end of a task [23].”
BackgroundAudio feedback system for engineering test satellite VII:
Nagai, Y., Kimura, S., Tsuchiya, S., & Iida, T. (1999) · SPIE Proceedings · doi:10.1117/12.369274 · Cited by 4 · cited 2×
“Reducing as such overall mental workload is correlated to fewer accidents and better performance [27].”
BackgroundResultBrickler, D., Babu, S. V., Bertrand, J., & Bhargava, A. (2018) · 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr.2018.8446227 · Cited by 4 · cited 2×
“Our findings show that haptic feedback leads to better performance which is supported by some studies [37], but contradicts others [45].”
BackgroundResult
Research works printed but not linked: 3
- simpson, 2013, Spatial audio display concepts supporting situation awareness for operators of unmanned aerial vehicles, Human Performance Situation Awareness and Automation Current Research and Trends, 2, 61
- richard, 1994, A comparison of haptic, visual and auditive force feedback for deformable virtual objects, Proc Internation Conf Autom Technol (ICAT), 49, 62
- aleotti, 2002, A Multi-modal User Interface for Remote Object Exploration in Teleopera-tion Systems
Cited by
Teleoperation methods and enhancement techniques for mobile robots: A comprehensive survey
Moniruzzaman, M., Rassau, A., Chai, D., & Islam, S. M. S. (2022) · Robotics and Autonomous Systems · doi:10.1016/j.robot.2021.103973 · Cited by 177
A Review of Human–Machine Cooperation in the Robotics Domain
Yang, C., Zhu, Y., & Chen, Y. (2022) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2021.3131684 · Cited by 128
A review on manipulation skill acquisition through teleoperation‐based learning from demonstration
Si, W., Wang, N., & Yang, C. (2021) · Cognitive Computation and Systems · doi:10.1049/ccs2.12005 · Cited by 140
“Extensive comparisons [30] have been done to show that regardless of task complexity, using multimodal interface could improve the performance.”
BackgroundInfluentialTechnologies for Multimodal Interaction in Extended Reality—A Scoping Review
Rakkolainen, I., Farooq, A., Kangas, J. A., Hakulinen, J., Rantala, J., Turunen, M., & Raisamo, R. (2021) · Multimodal Technologies and Interaction · doi:10.3390/mti5120081 · Cited by 136
“Finally, multimodal solutions can increase the level of immersion [228,229] and sometimes also task performance [229,230].”
BackgroundVibrotactile feedback in virtual motor learning: A systematic review
Islam, M. S., & Lim, S. I. (2022) · Applied Ergonomics · doi:10.1016/j.apergo.2022.103694 · Cited by 120
Metadata
- Authors
- Triantafyllidis, E., Mcgreavy, C., Gu, J., & Li, Z.
- Affiliations
- University of Edinburgh (GB)
- Venue
- IEEE Access
- Published
- 2020
- Volume
- 8
- Pages
- 78213-78227
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3019814707
- Cited by
- 105
- References
- 72
- Citations by year
- 2026: 10 · 2025: 17 · 2024: 25 · 2023: 17 · 2022: 16 · 2021: 20
- FWCI
- 6.52
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- Funders
- EPSRC Future AI and Robotics for Space (EP/R026092/1)
- Topics
- Physical Sciences › Engineering › Mechanical Engineering › Teleoperation and Haptic Systems
- Physical Sciences › Computer Science › Human-Computer Interaction › Virtual Reality Applications and Impacts
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Keywords
- multimodal interfaces, teleoperation, haptic feedback, auditory feedback, stereo vision, virtual reality headsets, manipulation tasks, mental workload, system usability, task performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.