Virtual Reality Musical Instruments: State of the Art, Design Principles, and Future Directions
The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
Abstract
The rapid development and availability of low-cost technologies have created a wide interest in virtual reality. In the field of computer music, the term “virtual musical instruments” has been used for a long time to describe software simulations, extensions of existing musical instruments, and ways to control them with new interfaces for musical expression. Virtual reality musical instruments (VRMIs) that include a simulated visual component delivered via a head-mounted display or other forms of immersive visualization have not yet received much attention. In this article, we present a field overview of VRMIs from the viewpoint of the performer. We propose nine design guidelines, describe evaluation methods, analyze case studies, and consider future challenges.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Philosophical conceptions of the self: implications for cognitive science.
Gallagher, S. (2000) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01417-5 · Cited by 2,881 · cited 3×
“The resulting sensation is called virtual body ownership, and several studies have shown that it is possible in VR to generate perceptual illusions of ownership over a virtual body seen from the first-person perspective, in other words, of a body that visually substitutes the person’s real body (Gallagher 2000). Kilte…”
BackgroundInfluentialPlace illusion and plausibility can lead to realistic behaviour in immersive virtual environments
Slater, M. (2009) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2009.0138 · Cited by 2,691 · cited 5×
“Particularly, Slater (2009) describes sensory motor contingencies as the set of actions a user will know how to perform in order to perceive something (e.g., moving one’s head and eyes to change gaze direction, kneeling to get a closer look at something on the ground, or turning one’s head to…”
BackgroundInfluentialSurround-screen projection-based virtual reality: the design and implementation of the CAVE
Cruz-Neira, C., Sandin, D. J., & DeFanti, T. A. (1993) · Proceedings of the 20th annual conference on Computer graphics and interactive techniques · doi:10.1145/166117.166134 · Cited by 2,327
3D User Interfaces: Theory and Practice
Bowman, D. A., Kruijff, E., LaViola, J. J., & Poupyrev, I. (2004) · Cited by 2,016
A discussion of cybersickness in virtual environments
LaViola, J. J. (2000) · ACM SIGCHI Bulletin · doi:10.1145/333329.333344 · Cited by 1,426 · cited 1×
“More generally, it is crucial to reduce system latency, since it is believed to increase cybersickness (LaViola 2000), the topic of the next principle.”
BackgroundNowak, K. L., & Biocca, F. (2003) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474603322761289 · Cited by 1,141
Sutherland, I. E. (1965) · Cited by 1,073
When second wave HCI meets third wave challenges
Bødker, S. (2006) · Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles · doi:10.1145/1182475.1182476 · Cited by 917 · cited 1×
“This layer focuses on higher design goals, embracing experience and “meaning-making” (Bødker 2006).”
BackgroundThe go-go interaction technique: non-linear mapping for direct manipulation in VR
Poupyrev, I., Billinghurst, M., Weghorst, S., & Ichikawa, T. (1996) · Proceedings of the 9th annual ACM symposium on User interface software and technology - UIST '96 · doi:10.1145/237091.237102 · Cited by 872
3-D sound for virtual reality and multimedia
Madole, D., & Begault, D. (1995) · Computer Music Journal · doi:10.2307/3680997 · Cited by 868 · cited 1×
“For virtual reality, 3-D sound is an especially relevant issue, because the location and motion of the visual virtual objects need to match the location and motion of the auditory objects (Begault 1994).”
Background
Research works printed but not linked: 21
- Berthaut F., 2014, IEEE VR Workshop: Sonic Interaction in Virtual Environments, 19
- Cook P., 2001, Proceedings of the International Conference on New Interfaces for Musical Expression, 3
- Cook P. R., 2009, Proceedings of the International Conference on New Interfaces for Musical Expression, 9, 218
- Fernandes A. S., 2016, IEEE Symposium on 3D User Interfaces, 201
- Fillwalk J., 2015, IEEE Symposium on 3D User Interfaces, 201
- Hamilton R., 2009, Proceedings of the AES 35th International Conference: Audio for Games, 1
- Heilbrun A., 1989, Whole Earth Review (64):108-119.
- Johnston A., 2011, Proceedings of the International Conference on New Interfaces for Musical Expression, 280
- Kohlrausch A., 1999, International Society for Optics and Photonics (SPIE) Proceedings on Human Vision and Electronic Imaging, 34
- Maki-Patola T., 2004, Proceedings of the International Conference on Digital Audio Effects, 11
- Mäki-Patola T., 2005, Proceedings of the International Conference on New Interfaces for Musical Expression, 11
- Marshall M. T., 2011, Proceedings of the International Conference on New Interfaces for Musical Expression, 399
- Moore A. G., 2015, Proceedings of the IEEE Symposium on 3D User Interfaces, 205
- Morreale F., 2014, Proceedings of the International Conference on New Interfaces for Musical Expression, 467
- Poupyrev I., 1996, Proceedings of the ACM Symposium on User Interface Software and Technology, 79
- Rodet X., 2005, Proceedings of the International Conference on New Interfaces for Musical Expression, 109
- Serafin S., 2006, Proceedings of the International Conference on New Interfaces for Musical Expression, 240
- Serafin S., 2015, IEEE Symposium on 3D User Interfaces, 207
- Wang G., 2014, Proceedings of the Joint International Computer Music Conference and the Sound and Music Computing Conference, 391
- Xiao X., 2014, Proceedings of the International Conference on New Interfaces for Musical Expression, 629
- Zielasko D., 2015, IEEE Symposium on 3D User Interfaces, 209
Cited by
Presence and Cybersickness in Virtual Reality Are Negatively Related: A Review
Weech, S., Kenny, S., & Barnett-Cowan, M. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00158 · Cited by 913
Internet of Musical Things: Vision and Challenges
Turchet, L., Fischione, C., Essl, G., Keller, D., & Barthet, M. (2018) · IEEE Access · doi:10.1109/access.2018.2872625 · Cited by 239
“A number of “virtual reality musical instruments” have been developed, which allow a user to play music within the virtual world (for a recent review see [121]).”
BackgroundQuestionnaire Measures and Physiological Correlates of Presence: A Systematic Review
Grassini, S., & Laumann, K. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.00349 · Cited by 209
Mobile virtual reality for musical genre learning in primary education
Innocenti, E. D., Geronazzo, M., Vescovi, D., Nordahl, R., Serafin, S., Ludovico, L. A., & Avanzini, F. (2019) · Computers & Education · doi:10.1016/j.compedu.2019.04.010 · Cited by 175
Self-citationTurchet, L., Hamilton, R., & Çamcı, A. (2021) · IEEE Access · doi:10.1109/access.2021.3052931 · Cited by 125
“In addition to their uses in performance situations, VRMIs also find use in music composition.”
BackgroundMethodInfluentialSonic Interactions in Virtual Reality: State of the Art, Current Challenges, and Future Directions
Serafin, S., Geronazzo, M., Erkut, C., Nilsson, N. C., & Nordahl, R. (2018) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2018.193142628 · Cited by 112
“28 Carefully designed sounds can be used to reproduce the acoustics of specific spaces, such as the Notre Dame Cathedral.”
MethodSelf-citationMusical Metaverse: vision, opportunities, and challenges
Turchet, L. (2023) · Personal and Ubiquitous Computing · doi:10.1007/s00779-023-01708-1 · Cited by 155
“A review of virtual reality musical instruments up to 2016 was surveyed by Serafin et al. [15].”
BackgroundSmart Musical Instruments: Vision, Design Principles, and Future Directions
Turchet, L. (2019) · IEEE Access · doi:10.1109/access.2018.2876891 · Cited by 94
A Survey on Visualizations for Musical Data
Khulusi, R., Kusnick, J., Meinecke, C., Gillmann, C., Focht, J., & Jänicke, S. (2020) · Computer Graphics Forum · doi:10.1111/cgf.13905 · Cited by 76
Ppali, S., Lalioti, V., Branch, B., Ang, C. S., Thomas, A. J., Wohl, B., & Covaci, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501922 · Cited by 64
“Specifcally, while some research eforts have focused on VR musical instruments for extending musical experiences (see [59] for a survey on this), the complex relationship musicians have with re-hearsal and performance spaces has been neglected.”
Fancourt, D. E., & Steptoe, A. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00778 · Cited by 52
Evaluating the effectiveness of mixed reality music instrument learning with the theremin
Johnson, D., Damian, D., & Tzanetakis, G. (2019) · Virtual Reality · doi:10.1007/s10055-019-00388-8 · Cited by 59
“For a more extensive overview of virtual reality music instruments (VRMIs), see the survey from Serafin et al. (2016).”
BackgroundYu, S., Liu, Q., Johnson‐Glenberg, M. C., Han, M., Ma, J., Ba, S., & Wu, L. (2023) · Computers & Education · doi:10.1016/j.compedu.2023.104764 · Cited by 51
Sonic Interactions in Virtual Environments
Geronazzo, M., & Serafin, S. (2022) · Human-computer interaction series · doi:10.1007/978-3-031-04021-4 · Cited by 57
Self-citationApplication of Immersive Virtual Reality Interactive Technology in Art Design Teaching
Ruan, Y. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/5987191 · Cited by 45
“Moreover, all these cannot limit it as [31] the best visual space cognitive research tool.”
BackgroundA Methodological Framework for Assessing Social Presence in Music Interactions in Virtual Reality
Kerrebroeck, B. V., Caruso, G., & Maes, P. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.663725 · Cited by 58
“They range from the search toward new virtual instruments (Honigman et al., 2013; Berthaut et al., 2014; Serafin et al., 2016; Hamilton, 2019) to new interactions and the development of interaction design principles (Deacon et al., 2016; Atherton and Wang, 2020).”
BackgroundAn Overview of FIR Filter Design in Future Multicarrier Communication Systems
Jiang, L., Zhang, H., Cheng, S., Lv, H., & Li, P. (2020) · Electronics · doi:10.3390/electronics9040599 · Cited by 38
Examples of use cases with Smart Instruments
Turchet, L., Benincaso, M., & Fischione, C. (2017) · Audio Mostly Conference (AM) · doi:10.1145/3123514.3123553 · Cited by 29
“For instance, virtual musical instruments [13] could be involved for this purpose, although latency issues currently represent a major obstacle to credible and sensitive interactions.”
BackgroundDoing vs. Being: A philosophy of design for artful VR
Atherton, J., & Wang, G. (2020) · Journal of New Music Research · doi:10.1080/09298215.2019.1705862 · Cited by 33
Towards The Internet Of Musical Things
Turchet, L., Fischione, C., & Barthet, M. (2017) · Institutional Research Information System (Università degli Studi di Trento) · doi:10.5281/zenodo.1401868 · Cited by 26
Social VR: A New Medium for Remote Communication and Collaboration
Li, J., Vinayagamoorthy, V., Schwartz, R., IJsselsteijn, W. A., Shamma, D. A., & Cesar, P. S. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3375160 · Cited by 34
“[24] argued that the design of VR musical instruments should enable social musical experiences, such as allowing the viewers to share the virtual space of the musicians.”
BackgroundOnline music learning based on digital multimedia for virtual reality
Sai, Y. (2022) · Interactive Learning Environments · doi:10.1080/10494820.2022.2127779 · Cited by 25
“These are complex interfaces that have evolved over the years, while VR interfaces provide mostly immersive visualization with simple controls compared to complex augmented reality tools to create a user-friendly application experience (Sera fi n et al., 2016).”
Immersive virtual reality application for intelligent manufacturing: Applications and art design
Lei, Y., Su, Z., He, X., & Cheng, C. (2022) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2023202 · Cited by 27
3D interaction techniques for musical expression
Berthaut, F. (2020) · Journal of New Music Research · doi:10.1080/09298215.2019.1706584 · Cited by 24
“Such instruments have been designated as Virtual Musical Instruments in Mulder (1998) and Pressing (1997), Immersive Virtual Musical Instruments (IVMI) in Berthaut, Zappi, and Maz-zanti (2014) or Virtual Reality Musical Instruments in Serafin, Erkut, Kojs, Nilsson, and Nordahl (2016).”
BackgroundMixed reality strategies for piano education
Amm, V., Chandran, K., Engeln, L., & McGinity, M. (2024) · Frontiers in Virtual Reality · doi:10.3389/frvir.2024.1397154 · Cited by 19
Díaz-López, L., Ortíz, J. T., & Contreras, C. P. (2019) · Digital Library Perspectives · doi:10.1108/dlp-08-2019-0034 · Cited by 17
Davanzo, N., & Avanzini, F. (2020) · IEEE Access · doi:10.1109/access.2020.3019978 · Cited by 16
“used to refer to DMIs that include a simulated visual component delivered via a head-mounted display or other forms of immersive visualization [90].”
MethodMusic Therapy in Virtual Reality for Autistic Children with Severe Learning Disabilities
Bauer, V., Adjorlu, A., Pedersen, L. B., Bouchara, T., & Serafin, S. (2023) · ACM Symposium on Virtual Reality Software and Technology (VRST) · doi:10.1145/3611659.3615713 · Cited by 20
“, VR enables the opportunity for creating virtual instruments withmagical interactions that do not follow real-world physical laws [34].”
BackgroundInfluentialSelf-citationSyukur, A., Andono, P. N., Hastuti, K., & Syarif, A. M. (2022) · Journal of Metaverse · doi:10.57019/jmv.1172928 · Cited by 18
Ubiquitous conformable systems for imperceptible computing
Fernandez, S. V., Sadat, D., Tasnim, F., Acosta, D., Schwendeman, L., Shahsavari, S., & Dağdeviren, C. (2021) · foresight · doi:10.1108/fs-07-2020-0067 · Cited by 16
No Strings Attached: Force and Vibrotactile Feedback in a Virtual Guitar Simulation
Passalenti, A., Paisa, R., Nilsson, N. C., Andersson, N. S., Fontana, F., Nordahl, R., & Serafin, S. (2019) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr.2019.8798168 · Cited by 14
“The availability of low-cost virtual reality (VR) hardware has entailed an increased interest in VR musical instruments (for a recent survey see [6]).”
BackgroundSelf-citationTai, K., Hong, J., Chen, K.-F., & Lin, C.-L. (2023) · Entertainment Computing · doi:10.1016/j.entcom.2023.100607 · Cited by 14
Gómez-Sirvent, J. L., Fernández-Sotos, A., Fernández‐Caballero, A., & Fernández-Sotos, D. (2024) · Virtual Reality · doi:10.1007/s10055-024-00947-8 · Cited by 14
“While there are tracking techniques that would have allowed them to see the instrument in the virtual environment (Serafin et al. 2016), their implementation is instrument-specific and would have been technically very costly to implement.”
Boem, A., & Turchet, L. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103231 · Cited by 12
A haptic feedback glove for virtual piano interaction
Fu, Y., Liu, J., Li, X., & Sun, X. (2025) · Virtual Reality & Intelligent Hardware · doi:10.1016/j.vrih.2024.07.001 · Cited by 12
Dong, W. (2021) · Journal of Sensors · doi:10.1155/2021/6119127 · Cited by 11
“Sensors are indispensable for the intelligent art design [1].”
BackgroundEvoluSon: Walking through an Interactive History of Music
Gaugne, R., Nouviale, F., Rioual, O., Chirat, A., Gohon, K., Goupil, V., Toutirais, M., Bossis, B., & Gouranton, V. (2017) · PRESENCE Virtual and Augmented Reality · doi:10.1162/pres_a_00298 · Cited by 11
Çamcı, A., & Granzow, J. (2019) · Leonardo Music Journal · doi:10.1162/lmj_a_01056 · Cited by 11
Sonic Interactions in Virtual Environments: The Egocentric Audio Perspective of the Digital Twin
Geronazzo, M., & Serafin, S. (2022) · Human-computer interaction series · doi:10.1007/978-3-031-04021-4_1 · Cited by 13
Self-citationGraphic Composition Art of Packaging Visual Communication Design Based on VR
Yan-na, W. (2022) · Wireless Communications and Mobile Computing · doi:10.1155/2022/4054207 · Cited by 10
“Digital 3D panorama technology, which is based on sequential pictures, stitches together static images to create a 360-degree panoramic image that accurately reproduces the real scene [1].”
BackgroundVirtual reality musical instruments
Serafin, S., Erkut, C., Kojs, J., Nordahl, R., & Nilsson, N. C. (2016) · Audio Mostly Conference (AM) · doi:10.1145/2986416.2986431 · Cited by 10
“These principles are enlisted in the sidebar; they are also converted to an evaluation framework, with three distinct layers in [12].”
BackgroundSelf-citationImmersive Io3MT Environments: Design Guidelines, Use Cases and Future Directions
Vieira, R., Wei, S., Röggla, T., Muchaluat-Saade, D. C., & Cesar, P. S. (2024) · IEEE International Symposium on the Internet of Sounds (IS2) · doi:10.1109/is262782.2024.10704141 · Cited by 9
“The combination of these factors with haptic elements, such as multisensory feedback and guidance, and an increase in the tangibility and physicality of the environment, can lead to new levels of abstraction, immersion, and imagination [15].”
BackgroundInfluentialCREATING AN INTERACTIVE MUSICAL EXPERIENCE FOR A CONCERT HALL
Antoshchuk, S. G., Kovalenko, M., & Sieck, J. (2018) · International Journal of Computing · doi:10.47839/ijc.17.3.1034 · Cited by 9
“A popular medium for creating virtual musical instruments is the use of Augmented Reality and object detection [4].”
MethodResearch of Digital Media Art based on Virtual Reality on Animation Design
Liu, B. (2019) · International Conference on Economics and Management, Education, Humanities and Social Sciences (EMEHSS) · doi:10.2991/emehss-19.2019.68 · Cited by 9
“The ideal virtual reality technology is able to imitate all human perceptions and achieve a high degree of simulation, so that users are completely immersed in the built virtual environment [10].”
BackgroundVrmin: Using Mixed Reality To Augment The Theremin For Musical Tutoring
Johnson, D., & Tzanetakis, G. (2017) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1176204 · Cited by 9
Online and In-Class Evaluation of a Music Theory E-Learning Platform
Pesek, M., Klavž, F., Šavli, P., & Marolt, M. (2022) · Applied Sciences · doi:10.3390/app12147296 · Cited by 11
“Gamification is included in various platforms, from mobile and web applications to hardware solutions [11] and virtual reality applications [12,13].”
BackgroundModular reality: Analogues of patching in immersive space
Palumbo, M., Zonta, A., & Wakefield, G. (2020) · Journal of New Music Research · doi:10.1080/09298215.2019.1706583 · Cited by 8
“In the terminology of Serafin et al. (2016), MSVR makes use of both ‘natural’ and ‘magical’ interactions, but these are built upon familiar spatial interaction metaphors.”
BackgroundResultInfluentialFirst Steps Towards Augmented Reality Interactive Electronic Music Production
Bauer, V., & Bouchara, T. (2021) · IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) · doi:10.1109/vrw52623.2021.00024 · Cited by 8
Graf, M., & Barthet, M. (2022) · New Interfaces for Musical Expression · doi:10.21428/92fbeb44.56ba9b93 · Cited by 10
Lim, D. H., Han, S. J., Oh, J., & Jang, C. S. (2018) · Advances in educational technologies and instructional design book series · doi:10.4018/978-1-5225-6322-8.ch015 · Cited by 8
A systematic review of technology-enhanced music learning models: Impacts on music education
Mohammadian, R., Ardakani, S. P., & Zavaraki, E. Z. (2026) · Computers and Education Open · doi:10.1016/j.caeo.2026.100354 · Cited by 10
Technologies as a Mediator between Creator and Audience in Postmodern Art Practices
Харченко, П., Чібалашвілі, А., Савчук, І., Sydorenko, V., & Хасанова, І. (2023) · BRAIN BROAD RESEARCH IN ARTIFICIAL INTELLIGENCE AND NEUROSCIENCE · doi:10.18662/brain/14.1/432 · Cited by 7
Synaesthetic Audio-Visual Sound Toys in Virtual Reality
Weinel, J. (2021) · Audio Mostly Conference (AM) · doi:10.1145/3478384.3478400 · Cited by 7
“sound toys in VR [6], the visual component of these experiences is typically used to provide graphical user interfaces and/or representations based on the appearance of real-world musical instruments.”
BackgroundSpatial Design Considerations for Interactive Audio in Virtual Reality
Deacon, T., & Barthet, M. (2022) · Human-computer interaction series · doi:10.1007/978-3-031-04021-4_6 · Cited by 10
Visualising Rave Music in Virtual Reality: Symbolic and interactive approaches
Weinel, J. (2020) · Electronic workshops in computing · doi:10.14236/ewic/eva2020.13 · Cited by 6
“This and other work in computer music research (e.g. Serafin et al. 2016) investigates various approaches to electronic music-making instruments using gestural interfaces, custom controllers and 'sound toys'.”
BackgroundGamelan Land: A Multiplayer Virtual Reality Game based on a Social Presence Approach
Syukur, A., Andono, P. N., & Syarif, A. M. (2023) · Journal of Metaverse · doi:10.57019/jmv.1334412 · Cited by 8
“The principles in the development of VRMIs formulated by [13] include a mapping between sound, visual, touch, and body position and movement (proprioception), minimum tolerance for delays in data transfer (latency) in visualization and sound, design inspiration but not replication based on real…”
Research on Music Visualization Based on Graphic Images and Mathematical Statistics
Li, W., & Li, J. (2020) · IEEE Access · doi:10.1109/access.2020.2999106 · Cited by 6
“The reflected probe light will also enter the photo detector along with the reference light through the optical path [18].”
BackgroundMygdanis, Y. (2022) · International Journal on Integrating Technology in Education · doi:10.5121/ijite.2022.11207 · Cited by 6
“Furthermore, connecting virtual instruments with sensors can contribute to multisensory experience formation through touch and movement, combined with augmented reality principles [33].”
BackgroundZappi, V., & Tatar, K. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2025.1575168 · Cited by 9
Tuning In to Intangibility : Reflections from My First 3 Years of Theremin Learning
Xiao, X., & Alaoui, S. F. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661584 · Cited by 6
“It has also inspired the design of new musical instruments [38, 50, 54].”
BackgroundCubing Sound: Designing a NIME for Head-mounted Augmented Reality
Wang, Y., & Martin, C. P. (2022) · New Interfaces for Musical Expression · doi:10.21428/92fbeb44.b540aa59 · Cited by 5
Design Aspects of a Virtual Reality Learning Environment to Assess Knowledge Transfer in Science
Steynberg, J., van Biljon, J. A., & Pilkington, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-63885-6_35 · Cited by 5
Go-Byar Based on Virtual Reality for the Learning Media of Gamelan
Dwipayana, K., Wirawan, I. M. A., & Sindu, I. G. P. (2019) · Jurnal Pendidikan Teknologi dan Kejuruan · doi:10.21831/jptk.v25i2.26182 · Cited by 5
“accessibility of low-cost technology has generated a wide interest in virtual reality [5].”
BackgroundSonic Interactions in Virtual Environments
Serafin, S. (2018) · Springer eBooks · doi:10.1007/978-3-319-08234-9_243-1 · Cited by 5
Self-citationImmersed in the Mind of a Music Composer: A VR Memoryscape for Exploring Archival Material
Stergiou, M., Servi, K., Gourdoglou, G., Kougioumtzian, L., Petousi, D., Katifori, A., Raheb, K. E., Boile, M., Kriezi, V., Merakos, S., et al. (2025) · Journal on Computing and Cultural Heritage · doi:10.1145/3737463 · Cited by 5
“Moreover, more and more researchers are proposing performing or practising instruments in VR, opening new avenues for exploring interactions having the benefits of visual feedback and immersion [39].”
Design and evaluation of a digitally active drum
Williams, P., & Overholt, D. (2020) · Personal and Ubiquitous Computing · doi:10.1007/s00779-020-01391-6 · Cited by 7
“[20].”
Playing by Ear: Designing for the Physical in a Sound-Based Virtual Reality Narrative
Harley, D., Tarun, A. P., Stinson, B. J., Tibu, T., & Mazalek, A. (2021) · International Conference on Tangible, Embedded, and Embodied Interaction (TEI) · doi:10.1145/3430524.3440635 · Cited by 4
“Although there is an important parallel history of music-based tangible applications (e.g., [20, 21, 23]), as well as musical instruments designed for virtual reality [39], we are interested in playful interactions rather than applications that aspire towards musical expression or mastery.”
BackgroundA Framework for the Design and Analysis of Mixed Reality Musical Instruments
Zellerbach, K. C., & Roberts, C. (2022) · New Interfaces for Musical Expression · doi:10.21428/92fbeb44.b2a44bc9 · Cited by 8
A Multiplayer Virtual Reality Game for Playing Gamelan Music in a Virtual Orchestra Mode
Syukur, A., Andono, P. N., & Syarif, A. M. (2023) · International Conference on Intelligent Cybernetics Technology & Applications (ICICyTA) · doi:10.1109/icicyta60173.2023.10428795 · Cited by 4
“The principles in the development of VRMIs formulated by [8] include: there is a mapping between sound, visual, touch, and body position and movement (proprioception); minimizing the tolerance for delays in data transfer (latency) in visualization and sound; not stuck on instrument replication but…”
BackgroundEncounter-Type Haptic Interfaces for Virtual Reality Musical Instruments
Boem, A., & Iwata, H. (2018) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr.2018.8446549 · Cited by 4
An Environment for Gestural Interaction with 3D Virtual Musical Instruments as an Educational Tool
Garoufis, C., Zlatintsi, A., Kritsis, K., Filntisis, P. P., Katsouros, V., & Maragos, P. A. (2019) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco.2019.8902604 · Cited by 4
“A virtual musical instrument [17] is defined as a non-physical object that constitutes a simulation of an existing musical instrument, focusing on sonic emulation, while a virtual reality musical instrument extends on that, by providing visual as well as auditory feedback.”
BackgroundSonic Interactions in Virtual Environments
Serafin, S. (2024) · Springer eBooks · doi:10.1007/978-3-031-23161-2_243 · Cited by 4
Self-citationAssessing Impacts of Immersive Virtual Reality-Based Design Reviews on Learners’ Self-Efficacy
Castronovo, F., RazaviAlavi, S., Abdulrahman, A., Saiba, M. R., & Martínez, P. (2023) · Proceedings e report · doi:10.36253/979-12-215-0289-3.21 · Cited by 3
“It Takes Two” - Shared and Collaborative Virtual Musical Instruments in the Musical Metaverse
Boem, A., Dziwis, D. T., Tomasetti, M., Etezazi, S., & Turchet, L. (2024) · IEEE International Symposium on the Internet of Sounds (IS2) · doi:10.1109/is262782.2024.10704079 · Cited by 8
“…such capabilities of VEs together with the possibilities of real-time digital sound synthesis and Virtual Reality (VR) hardware provides the basis for the development not only of novel Virtual Musical Instruments (VMIs) [17] [18] but also for live performances [19] and music-making in general [20].”
BackgroundZappi, V., Mazzanti, D., & Berthaut, F. (2022) · Human-computer interaction series · doi:10.1007/978-3-031-04021-4_13 · Cited by 4
Spatial Audio Mixing in Virtual Reality
Bargum, A. R., Kristjansson, O. I., Babó, P., Nielsen, R. E. W., Mosen, S. R., & Serafin, S. (2022) · Human-computer interaction series · doi:10.1007/978-3-031-04021-4_9 · Cited by 3
Self-citationSyahidi, A. A., & Kiyokawa, K. (2023) · International Conference on Information Technology and Computing (ICITCOM) · doi:10.1109/icitcom60176.2023.10442960 · Cited by 3
“According to [3], VRMI in computer music refers to simulation software, enhancing existing instruments, and novel interfaces for musical expression.”
BackgroundReal-Time Motion Capture Analysis And Music Interaction With The Modosc Descriptor Library
Visi, F., & Dahl, L. (2018) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1302706 · Cited by 3
A Framework for Development of Serious Game for Learning Musical Instrument
Taufiq, M., & Hendradjaya, B. (2018) · International Conference on Data and Software Engineering · doi:10.1109/icodse.2018.8705885 · Cited by 3
“Games and Simulation-Based Education Framework [11] √ √ √ √”
BackgroundInfluentialA Meta-interactive Compositional Approach that Fosters Musical Emergence through Ludic Expressivity
Escarce, M., Martins, G. R., Marcolino, L. S., & Rubegni, E. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3474689 · Cited by 5
“[59], the NIME (New Interfaces for Musical Expression) community has not shown the proper attention to the imagetic side of VRMIs.”
BackgroundHedges, J., Sazdov, R., & Johnston, A. (2025) · Convergence The International Journal of Research into New Media Technologies · doi:10.1177/13548565251352078 · Cited by 2
Resin: a Vocal Tract Resonances and Head Based Accessible Digital Musical Instrument
Davanzo, N., & Avanzini, F. (2021) · Audio Mostly Conference (AM) · doi:10.1145/3478384.3478403 · Cited by 2
“It has also been used to control virtual reality instruments [8, 24], and augmented acoustic instruments [14].”
MethodTouching the Untouchable: Playing the Virtual Glass Harmonica
Pedersen, A., Jørgensen, M., & Serafin, S. (2024) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-55312-7_16 · Cited by 2
Self-citationMotion Controllers, Sound, and Music in Video Games: State of the Art and Research Perspectives
Visi, F., & Faasch, F. (2018) · International series on computer entertainment and media technology · doi:10.1007/978-3-319-72272-6_8 · Cited by 2
Xie, N., Cai, X., Li, S., Lu, Y., Lou, M., & Shen, H. T. (2018) · SIGGRAPH Asia Posters · doi:10.1145/3283289.3283364 · Cited by 2
“DT-Zheng system provides an environment of virtual reality musical instruments (VRMIs) [Serafin et al. 2016] called NingXian with three modes.”
BackgroundThe Effect of Control-Display Ratio on User Experience in Immersive Virtual Musical Instruments
Reynaert, V., Berthaut, F., Rekik, Y., & Grisoni, L. (2021) · New Interfaces for Musical Expression · doi:10.21428/92fbeb44.c47be986 · Cited by 2
Participatory Design Workshop to Create a Virtual Reality Musical Instrument Based on Tumbuk Kalang
Baharin, H., Norowi, N. M., Saffian, K. A., & Sim, Y. E. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-7339-2_13 · Cited by 1
Hsieh, Y., Lin, J.-J., Su, M., & Lin, W.-J. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3535282 · Cited by 1
Zempo, K., Kashiwabara, R., Wakatsuki, N., & Mizutani, K. (2025) · IEEE Access · doi:10.1109/access.2025.3565283 · Cited by 1
“It is also known that the presentation of virtual tactile sensations in VR space improves immersion while listening to music [26], [27].”
Computer-Human Interaction in Music Composition Through Petri Nets
Baratè, A., Haus, G. M., & Ludovico, L. A. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-67108-2_4 · Cited by 1
Metaverse: Fission the architecture
Zaman, A., Abir, M. R., Reaj, M. A. H., & Sakib, T. H. (2024) · International Journal of Science and Research Archive · doi:10.30574/ijsra.2024.12.1.0808 · Cited by 1
Exploring Musical Affordance of a Virtual Reality Malay Gamelan Through Qualitative Study
Saffian, K. A., & Norowi, N. M. (2025) · Journal of Metaverse · doi:10.57019/jmv.1609739 · Cited by 1
Boem, A., Tomasetti, M., Gabriele, A., Scipio, A. D., Micalizzi, A., & Turchet, L. (2025) · Digital Creativity · doi:10.1080/14626268.2025.2542146 · Cited by 1
“Moreover, these dimensions represent the core areas where our participant groups engage professionally and where immersive technologies such as AR/MR/VR can provide a positive impact (Gómez-Sirvent et al. 2024; Serafin et al. 2016; Turchet, Hamilton, and Çamci 2021).”
Metaphysical instruments: prototypes for hybrid and live music-making
Trujillo-Pisanty, D., Olguin-Simon, H., Spitalier-Tron, N., Ferrari-Garcia, P., Fragoso-Terreros, J., Lobo-Barrera, A., Peña-Rios, X., Pous-Pierson, P., Toledo-Galindo, F., Torres-Cazares, J., et al. (2023) · Academia Press eBooks · doi:10.26530/9789401496476-048 · Cited by 1
An AI-Mediated VR Sound Installation
Santini, G., & Chen, Z. (2024) · Lecture notes in civil engineering · doi:10.1007/978-981-97-0621-1_27 · Cited by 1
Artistic Practices in the Context of the Evolution from Web 1.0 to Web 3.0
Чібалашвілі, А., Безугла, Р. І., Харченко, П., Severynova, M. Y., & Korenyuk, Y. (2024) · Studia Universitatis Babeş-Bolyai Musica · doi:10.24193/subbmusica.2024.spiss2.08 · Cited by 1
Sound Blocks VR: An Accessible Virtual Reality Musical Instrument
Gioiosa, M., Avanzini, F., Ludovico, L. A., Brambilla, S., & Ripamonti, L. A. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-83845-3_19 · Cited by 1
Musical Brush: Exploring Creativity Through an AR-Based Tool for Sketching Music and Drawings
Valer, R., Schramm, R., & Nedel, L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61864-3_11 · Cited by 1
Developments in Implicit Measurements
Ziepert, B., de Vries, P. W., Ufkes, E. G., Toet, A., Anne-Marie, B., Barski, J. J., Spink, A., Riedel, G., & Roy, R. N. (2022) · Frontiers research topics · doi:10.3389/978-2-88974-342-1 · Cited by 1
Meta-interactivity and Playful Approaches for Musical Composition
Escarce, M. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3503819 · Cited by 1
“[2, 24]; (ii) studies in interfaces capable to foster efficient devices for the”
Background
Metadata
- Authors
- Serafin, S., Erkut, C., Kojs, J., Nilsson, N. C., & Nordahl, R.
- Affiliations
- Aalborg University (DK)
- University of Miami (US)
- Venue
- Computer Music Journal
- Publisher
- MIT Press
- Published
- 2016-09
- Volume
- 40
- Issue
- 3
- Pages
- 22-40
- ISSN
- 0148-9267 (ISSN-L)
- 1531-5169
- OpenAlex
- W2563502399
- Cited by
- 139
- References
- 74
- Citations by year
- 2026: 11 · 2025: 13 · 2024: 14 · 2023: 17 · 2022: 25 · 2021: 16 · 2020: 15 · 2019: 10 · 2018: 13 · 2017: 4 · 2016: 1
- FWCI
- 6.34
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Computer Music Journal (publishedVersion)
- Project Muse (Johns Hopkins University) (submittedVersion)
- VBN Forskningsportal (Aalborg Universitet) (submittedVersion)
- Topics
- Social Sciences › Arts and Humanities › Music › Music Technology and Sound Studies
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Physical Sciences › Computer Science › Human-Computer Interaction › Virtual Reality Applications and Impacts
- Keywords
- virtual musical instruments, immersive visualization, head-mounted displays, musical instrument design, design guidelines, performer perspective, musical expression, assessment methods, computer music
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.