The Rocketbox Library and the Utility of Freely Available Rigged Avatars
Open accessThe DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
As part of the open sourcing of the Microsoft Rocketbox avatar library for research and academic purposes, here we discuss the importance of rigged avatars for the Virtual and Augmented Reality (VR, AR) research community. Avatars, virtual representations of humans, are widely used in VR applications. Furthermore many research areas ranging from crowd simulation to neuroscience, psychology or sociology have used avatars to investigate new theories or to demonstrate how they influence human performance and interactions. We divide this paper in two main parts: the first one gives an overview of the different methods available to create and animate avatars. We cover the current main alternatives for face and body animation as well introduce upcoming capture methods. The second part presents the scientific evidence of the utility of using rigged avatars for embodiment but also for applications such as crowd simulation and entertainment. All in all this paper attempts to convey why rigged avatars will be key to the future of VR and its wide adoption.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714 · cited 1×
“Deep learning methods can also generate completely new avatars that are not representations of existing people, by using adversarial networks (Karras et al., 2019).”
BackgroundRealtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539 · cited 1×
“Research is examining the use of additional sensors and tracking systems such as external (Spanlang et al., 2014; Cao et al., 2017; Mehta et al., 2017) or wearable cameras (Ahuja et al.”
BackgroundBem, D. J. (1972) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60024-6 · Cited by 5,241 · cited 1×
“Self-perception theory argues that people often infer their own attitudes and beliefs from observing themselves—for example their facial expressions or styles of dress—as if from a third party (Bem, 1972).”
BackgroundRubber hands ‘feel’ touch that eyes see
Botvinick, M., & Cohen, J. (1998) · Nature · doi:10.1038/35784 · Cited by 4,410 · cited 4×
“This body ownership effect that was first demonstrated with a rubber hand (Botvinick and Cohen, 1998), has now been replicated in a large number of instances with virtual avatars, (for a review see the Slater and Sanchez-Vives, 2016; Gonzalez-Franco and Lanier, 2017).”
BackgroundInfluentialPlace illusion and plausibility can lead to realistic behaviour in immersive virtual environments
Slater, M. (2009) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2009.0138 · Cited by 2,691
Self-citationA Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms
Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.19 · Cited by 2,457
The Proteus Effect: The Effect of Transformed Self-Representation on Behavior
Yee, N., & Bailenson, J. (2007) · Human Communication Research · doi:10.1111/j.1468-2958.2007.00299.x · Cited by 2,126
Self-citationEnhancing Our Lives with Immersive Virtual Reality
Slater, M., & Sanchez-Vives, M. V. (2016) · Frontiers in Robotics and AI · doi:10.3389/frobt.2016.00074 · Cited by 1,993
Self-citationFrom presence to consciousness through virtual reality.
Sanchez-Vives, M. V., & Slater, M. (2005) · Nature reviews. Neuroscience · doi:10.1038/nrn1651 · Cited by 1,831
Self-citationVideo ergo sum: manipulating bodily self-consciousness
Lenggenhager, B., Tadi, T., Metzinger, T., & Blanke, O. (2007) · Science · doi:10.1126/science.1143439 · Cited by 1,265
Multisensory brain mechanisms of bodily self-consciousness.
Blanke, O. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3292 · Cited by 1,147
First person experience of body transfer in virtual reality
Slater, M., Spanlang, B., Sanchez-Vives, M. V., & Blanke, O. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010564 · Cited by 1,090
Self-citationCausal Inference in Multisensory Perception
Körding, K. P., Beierholm, U., Ma, W. J., Quartz, S., Tenenbaum, J. B., & Shams, L. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000943 · Cited by 1,069 · cited 1×
“More recently, computational models developed in the framework of causal inference (Körding et al., 2007; Shams and Beierholm, 2010) described the onset of illusory ownership as the result of an “inference process” in which the brain associates all the available sensory information about the body…”
MethodPutting yourself in the skin of a black avatar reduces implicit racial bias
Peck, T. C., Seinfeld, S., Aglioti, S. M., & Slater, M. (2013) · Consciousness and Cognition · doi:10.1016/j.concog.2013.04.016 · Cited by 967 · cited 3×
“Several studies have shown that in neutral or positive social circumstances embodiment of White people in a Black virtual body results in a reduction in implicit racial bias, measured using the Implicit Association Test (Peck et al., 2013).”
BackgroundMethodResultInfluentialSelf-citationWhat is embodiment? A psychometric approach
Longo, M. R., Schüür, F., Kammers, M. P. M., Tsakiris, M., & Haggard, P. (2008) · Cognition · doi:10.1016/j.cognition.2007.12.004 · Cited by 956
If I were you: Perceptual illusion of body swapping
Petkova, V. I., & Ehrsson, H. H. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003832 · Cited by 953 · cited 1×
“Petkova and Ehrsson (2008) showed how the techniques used for the rubber-hand illusion could be applied to the whole body thus producing a “body ownership illusion” of a mannequin’s body and this also has been reproduced in VR (Slater et al., 2010b; Yuan and Steed, 2010).”
BackgroundProjecting sensations to external objects: evidence from skin conductance response
Armel, K. C., & Ramachandran, V. S. (2003) · Proceedings of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rspb.2003.2364 · Cited by 854
Banakou, D., Groten, R., & Slater, M. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1306779110 · Cited by 695 · cited 1×
“As a result, they self-identify more with child-like attributes and see the surrounding world as larger (Banakou et al., 2013; Tajadura-Jiménez et al., 2017).”
BackgroundSelf-citationWaltemate, T., Gall, D., Roth, D., Botsch, M., & Latoschik, M. E. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2794629 · Cited by 551 · cited 1×
“Scanning is particularly useful if an avatar needs to be modeled on existing real people (Waltemate et al., 2018).”
BackgroundOn the other hand: Dummy hands and peripersonal space
Makin, T. R., Holmes, N. P., & Ehrsson, H. H. (2008) · Behavioural Brain Research · doi:10.1016/j.bbr.2008.02.041 · Cited by 530
Extending body space in immersive virtual reality: a very long arm illusion
Kilteni, K., Normand, J.-M., Sanchez-Vives, M. V., & Slater, M. (2012) · PLoS ONE · doi:10.1371/journal.pone.0040867 · Cited by 499 · cited 1×
“…et al., 2007; Blanke, 2012; Maselli and Slater, 2014), as well as by modulating the size and/or shape of the virtual body that interacts with this peripersonal space (Abtahi et al., 2019), for example by having an elongated virtual arm (Kilteni et al., 2012b; Feuchtner and Müller, 2017).”
BackgroundSelf-citationAutomatic rigging and animation of 3D characters
Baran, I., & Popović, J. (2007) · ACM SIGGRAPH 2007 papers · doi:10.1145/1275808.1276467 · Cited by 495 · cited 1×
“There is a trend toward easier tools and automatic systems for rigging meshes and avatars (Baran and Popovi´c, 2007; Feng et al., 2015).”
MethodAutomatic rigging and animation of 3D characters
Baran, I., & Popović, J. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276467 · Cited by 455
Virtual hand illusion induced by visuomotor correlations
Sanchez-Vives, M. V., Spanlang, B., Frisoli, A., Bergamasco, M., & Slater, M. (2010) · PLoS ONE · doi:10.1371/journal.pone.0010381 · Cited by 441 · cited 1×
“The illusion induced by visuomotor synchrony, where the body moves synchronously with the movements of the person, is typically more vivid than those triggered by visuotactile synchrony, where the virtual body is passively touched synchronously with real touch (GonzalezFranco et al., 2010; Sanchez-Vives et al., 2010;…”
BackgroundGarau, M., Slater, M., Vinayagamoorthy, V., Brogni, A., Steed, A., & Sasse, M. A. (2003) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642703 · Cited by 441
Self-citationCausal inference in perception
Shams, L., & Beierholm, U. R. (2010) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2010.07.001 · Cited by 403 · cited 1×
“More recently, computational models developed in the framework of causal inference (Körding et al., 2007; Shams and Beierholm, 2010) described the onset of illusory ownership as the result of an “inference process” in which the brain associates all the available sensory information about the body…”
MethodThreatening a rubber hand that you feel is yours elicits a cortical anxiety response.
Ehrsson, H. H., Wiech, K., Weiskopf, N., Dolan, R. J., & Passingham, R. E. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0610011104 · Cited by 390 · cited 1×
“For most people, after about 1 minute of this stimulation and visuo-tactile correlation, proprioception shifts to the rubber hand, and if the rubber hand is threatened there is a strong physiological response (Armel and Ramachandran, 2003) and corresponding brain activation (Ehrsson et al., 2007).”
BackgroundChanging bodies changes minds: Owning another body affects social cognition
Maister, L., Slater, M., Sanchez-Vives, M. V., & Tsakiris, M. (2015) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2014.11.001 · Cited by 384 · cited 2×
“…perspective, and undergoing the illusion of having one’s own physical body (concealed from view) substituted by the virtual body seen in its place and moving accordingly, could further leverage the impact of self-avatars on implicit attitudes, social behavior, and cognition (Maister et al., 2015).”
BackgroundResultSelf-citationDrumming in Immersive Virtual Reality: The Body Shapes the Way We Play
Kilteni, K., Bergstrom, I., & Slater, M. (2013) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2013.29 · Cited by 326
Self-citationPerception of Body Ownership Is Driven by Bayesian Sensory Inference
Samad, M., Chung, A. J., & Shams, L. (2015) · PLOS ONE · doi:10.1371/journal.pone.0117178 · Cited by 315 · cited 1×
“…result of an “inference process” in which the brain associates all the available sensory information about the body (visual cues from the virtual, together with somatosensory or proprioceptive cues from the physical body) to a single origin: the own body (Kilteni et al., 2015; Samad et al., 2015).”
BackgroundEmbodying self-compassion within virtual reality and its effects on patients with depression
Falconer, C. J., Rovira, A., King, J. A., Gilbert, P., Antley, A., Fearon, P., Ralph, N., Slater, M., & Brewin, C. R. (2016) · BJPsych Open · doi:10.1192/bjpo.bp.115.002147 · Cited by 312
Self-citationPetkova, V. I., Khoshnevis, M., & Ehrsson, H. H. (2011) · Frontiers in Psychology · doi:10.3389/fpsyg.2011.00035 · Cited by 311 · cited 1×
“First-person perspective over the virtual body that visually substitutes the real body in VR is a prerequisite for this type of body ownership illusion (Petkova et al., 2011; Maselli and Slater, 2014).”
BackgroundBanakou, D., Kishore, S., & Slater, M. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.00917 · Cited by 305 · cited 2×
“Even more interestingly, self-avatars can even help users to better perform cognitive tasks (Steed et al., 2016b; Banakou et al., 2018), modify implicit racial bias (Groom et al., 2009; Peck et al., 2013; Banakou et al., 2016; Hasler et al., 2017; Salmanowitz, 2018) or even change, for example,…”
BackgroundSelf-citationKokkinara, E., & Slater, M. (2014) · Perception · doi:10.1068/p7545 · Cited by 300 · cited 2×
“The illusion induced by visuomotor synchrony, where the body moves synchronously with the movements of the person, is typically more vivid than those triggered by visuotactile synchrony, where the virtual body is passively touched synchronously with real touch (GonzalezFranco et al., 2010; Sanchez-Vives et al., 2010;…”
BackgroundSelf-citationOsimo, S. A., Pizarro, R., Spanlang, B., & Slater, M. (2015) · Scientific Reports · doi:10.1038/srep13899 · Cited by 242 · cited 2×
“…Gonzalez-Franco et al., 2019b), to explore the effects of self-compassion (Falconer et al., 2014, 2016), crowd simulation (Pelechano et al., 2007), or even experiencing the world from the embodied viewpoint of another (Osimo et al., 2015; Hamilton-Giachritsis et al., 2018; Seinfeld et al., 2018 ).”
BackgroundSelf-citationBody ownership causes illusory self-attribution of speaking and influences subsequent real speaking
Banakou, D., & Slater, M. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1414936111 · Cited by 223 · cited 1×
“It was found that participants have agency over the speaking, and the way they themselves speak later is influenced by how their self-avatar spoke (Banakou and Slater, 2014).”
BackgroundSelf-citationExperiencing ownership over a dark-skinned body reduces implicit racial bias
Maister, L., Sebanz, N., Knoblich, G., & Tsakiris, M. (2013) · Cognition · doi:10.1016/j.cognition.2013.04.002 · Cited by 221 · cited 2×
“Similar results had been found with the rubber-hand illusion over a black rubber hand (Maister et al., 2013), and these results and explanations for them were discussed in (Maister et al., 2015).”
ResultHow the brain represents the body: insights from neurophysiology and psychology.
Graziano, M. S. A., & Botvinick, M. M. (2002) · Common Mechanisms in Perception and Action · doi:10.1093/oso/9780198510697.003.0006 · Cited by 174
Embodying Compassion: A Virtual Reality Paradigm for Overcoming Excessive Self-Criticism
Falconer, C. J., Slater, M., Rovira, A., King, J. A., Gilbert, P., Antley, A., & Brewin, C. R. (2014) · PLoS ONE · doi:10.1371/journal.pone.0111933 · Cited by 155
Self-citationAn experimental study of a virtual reality counselling paradigm using embodied self-dialogue
Slater, M., Neyret, S., Johnston, T., Iruretagoyena, G., Crespo, M. Á. D. L. C., Alabèrnia-Segura, M., Spanlang, B., & Feixas, G. (2019) · Scientific Reports · doi:10.1038/s41598-019-46877-3 · Cited by 155
Self-citationMatamala-Gomez, M., Diaz Gonzalez, A. M., Slater, M., & Sanchez-Vives, M. V. (2019) · The Journal of Pain · doi:10.1016/j.jpain.2018.12.001 · Cited by 118
Self-citationBerger, C. C., Gonzalez-Franco, M., Ofek, E., & Hinckley, K. (2018) · Science Robotics · doi:10.1126/scirobotics.aar7010 · Cited by 109 · cited 1×
“Interestingly, and highly useful as a control, if there is asynchrony or incongruence between sensory inputs (either in space or in time) or in sensorimotor correlations, the illusion breaks (Berger et al., 2018; Gonzalez-Franco and Berger, 2019).”
BackgroundSelf-citationExtending the Body for Interaction with Reality
Feuchtner, T., & Müller, J. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025689 · Cited by 85
The relationship between virtual body ownership and temperature sensitivity
Llobera, J., Sanchez-Vives, M. V., & Slater, M. (2013) · Journal of The Royal Society Interface · doi:10.1098/rsif.2013.0300 · Cited by 81
Self-citation
Metadata
- Authors
- Gonzalez-Franco, M., Ofek, E., Pan, Y., Antley, A., Steed, A., Spanlang, B., Maselli, A., Banakou, D., Pelechano, N., Orts-Escolano, S., Orvalho, V., Trutoiu, L., Wojcik, M., Sanchez-Vives, M. V., Bailenson, J., Slater, M., & Lanier, J.
- Venue
- Frontiers in Virtual Reality
- Publisher
- Frontiers Media SA
- Published
- 2020-11-03
- Volume
- 1
- ISSN
- 2673-4192 (ISSN-L)
- OpenAlex
- W3087311975
- Cited by
- 190
- References
- 44
- FWCI
- 18.79
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.