Virtual Reality Musical Instruments: State of the Art, Design Principles, and Future Directions
The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
Abstract
The rapid development and availability of low-cost technologies have created a wide interest in virtual reality. In the field of computer music, the term “virtual musical instruments” has been used for a long time to describe software simulations, extensions of existing musical instruments, and ways to control them with new interfaces for musical expression. Virtual reality musical instruments (VRMIs) that include a simulated visual component delivered via a head-mounted display or other forms of immersive visualization have not yet received much attention. In this article, we present a field overview of VRMIs from the viewpoint of the performer. We propose nine design guidelines, describe evaluation methods, analyze case studies, and consider future challenges.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Philosophical conceptions of the self: implications for cognitive science.
Gallagher, S. (2000) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(99)01417-5 · Cited by 2,881 · cited 3×
“The resulting sensation is called virtual body ownership, and several studies have shown that it is possible in VR to generate perceptual illusions of ownership over a virtual body seen from the first-person perspective, in other words, of a body that visually substitutes the person’s real body (Gallagher 2000). Kilte…”
BackgroundInfluentialPlace illusion and plausibility can lead to realistic behaviour in immersive virtual environments
Slater, M. (2009) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2009.0138 · Cited by 2,691 · cited 5×
“Particularly, Slater (2009) describes sensory motor contingencies as the set of actions a user will know how to perform in order to perceive something (e.g., moving one’s head and eyes to change gaze direction, kneeling to get a closer look at something on the ground, or turning one’s head to…”
BackgroundInfluentialSurround-screen projection-based virtual reality: the design and implementation of the CAVE
Cruz-Neira, C., Sandin, D. J., & DeFanti, T. A. (1993) · Proceedings of the 20th annual conference on Computer graphics and interactive techniques · doi:10.1145/166117.166134 · Cited by 2,327
3D User Interfaces: Theory and Practice
Bowman, D. A., Kruijff, E., LaViola, J. J., & Poupyrev, I. (2004) · Cited by 2,016
A discussion of cybersickness in virtual environments
LaViola, J. J. (2000) · ACM SIGCHI Bulletin · doi:10.1145/333329.333344 · Cited by 1,426 · cited 1×
“More generally, it is crucial to reduce system latency, since it is believed to increase cybersickness (LaViola 2000), the topic of the next principle.”
BackgroundNowak, K. L., & Biocca, F. (2003) · Presence: Teleoperators and Virtual Environments · doi:10.1162/105474603322761289 · Cited by 1,141
Sutherland, I. E. (1965) · Cited by 1,073
When second wave HCI meets third wave challenges
Bødker, S. (2006) · Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles · doi:10.1145/1182475.1182476 · Cited by 917 · cited 1×
“This layer focuses on higher design goals, embracing experience and “meaning-making” (Bødker 2006).”
BackgroundThe go-go interaction technique: non-linear mapping for direct manipulation in VR
Poupyrev, I., Billinghurst, M., Weghorst, S., & Ichikawa, T. (1996) · Proceedings of the 9th annual ACM symposium on User interface software and technology - UIST '96 · doi:10.1145/237091.237102 · Cited by 872
3-D sound for virtual reality and multimedia
Madole, D., & Begault, D. (1995) · Computer Music Journal · doi:10.2307/3680997 · Cited by 868 · cited 1×
“For virtual reality, 3-D sound is an especially relevant issue, because the location and motion of the visual virtual objects need to match the location and motion of the auditory objects (Begault 1994).”
Background
Research works printed but not linked: 21
- Berthaut F., 2014, IEEE VR Workshop: Sonic Interaction in Virtual Environments, 19
- Cook P., 2001, Proceedings of the International Conference on New Interfaces for Musical Expression, 3
- Cook P. R., 2009, Proceedings of the International Conference on New Interfaces for Musical Expression, 9, 218
- Fernandes A. S., 2016, IEEE Symposium on 3D User Interfaces, 201
- Fillwalk J., 2015, IEEE Symposium on 3D User Interfaces, 201
- Hamilton R., 2009, Proceedings of the AES 35th International Conference: Audio for Games, 1
- Heilbrun A., 1989, Whole Earth Review (64):108-119.
- Johnston A., 2011, Proceedings of the International Conference on New Interfaces for Musical Expression, 280
- Kohlrausch A., 1999, International Society for Optics and Photonics (SPIE) Proceedings on Human Vision and Electronic Imaging, 34
- Maki-Patola T., 2004, Proceedings of the International Conference on Digital Audio Effects, 11
- Mäki-Patola T., 2005, Proceedings of the International Conference on New Interfaces for Musical Expression, 11
- Marshall M. T., 2011, Proceedings of the International Conference on New Interfaces for Musical Expression, 399
- Moore A. G., 2015, Proceedings of the IEEE Symposium on 3D User Interfaces, 205
- Morreale F., 2014, Proceedings of the International Conference on New Interfaces for Musical Expression, 467
- Poupyrev I., 1996, Proceedings of the ACM Symposium on User Interface Software and Technology, 79
- Rodet X., 2005, Proceedings of the International Conference on New Interfaces for Musical Expression, 109
- Serafin S., 2006, Proceedings of the International Conference on New Interfaces for Musical Expression, 240
- Serafin S., 2015, IEEE Symposium on 3D User Interfaces, 207
- Wang G., 2014, Proceedings of the Joint International Computer Music Conference and the Sound and Music Computing Conference, 391
- Xiao X., 2014, Proceedings of the International Conference on New Interfaces for Musical Expression, 629
- Zielasko D., 2015, IEEE Symposium on 3D User Interfaces, 209
Cited by
Presence and Cybersickness in Virtual Reality Are Negatively Related: A Review
Weech, S., Kenny, S., & Barnett-Cowan, M. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00158 · Cited by 913
Internet of Musical Things: Vision and Challenges
Turchet, L., Fischione, C., Essl, G., Keller, D., & Barthet, M. (2018) · IEEE Access · doi:10.1109/access.2018.2872625 · Cited by 239
“A number of “virtual reality musical instruments” have been developed, which allow a user to play music within the virtual world (for a recent review see [121]).”
BackgroundQuestionnaire Measures and Physiological Correlates of Presence: A Systematic Review
Grassini, S., & Laumann, K. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.00349 · Cited by 209
Mobile virtual reality for musical genre learning in primary education
Innocenti, E. D., Geronazzo, M., Vescovi, D., Nordahl, R., Serafin, S., Ludovico, L. A., & Avanzini, F. (2019) · Computers & Education · doi:10.1016/j.compedu.2019.04.010 · Cited by 175
Self-citationTurchet, L., Hamilton, R., & Çamcı, A. (2021) · IEEE Access · doi:10.1109/access.2021.3052931 · Cited by 125
“In addition to their uses in performance situations, VRMIs also find use in music composition.”
BackgroundMethodInfluential
Metadata
- Authors
- Serafin, S., Erkut, C., Kojs, J., Nilsson, N. C., & Nordahl, R.
- Affiliations
- Aalborg University (DK)
- University of Miami (US)
- Venue
- Computer Music Journal
- Publisher
- MIT Press
- Published
- 2016-09
- Volume
- 40
- Issue
- 3
- Pages
- 22-40
- ISSN
- 0148-9267 (ISSN-L)
- 1531-5169
- OpenAlex
- W2563502399
- Cited by
- 139
- References
- 74
- Citations by year
- 2026: 11 · 2025: 13 · 2024: 14 · 2023: 17 · 2022: 25 · 2021: 16 · 2020: 15 · 2019: 10 · 2018: 13 · 2017: 4 · 2016: 1
- FWCI
- 6.34
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Computer Music Journal (publishedVersion)
- Project Muse (Johns Hopkins University) (submittedVersion)
- VBN Forskningsportal (Aalborg Universitet) (submittedVersion)
- Topics
- Social Sciences › Arts and Humanities › Music › Music Technology and Sound Studies
- Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
- Physical Sciences › Computer Science › Human-Computer Interaction › Virtual Reality Applications and Impacts
- Keywords
- virtual musical instruments, immersive visualization, head-mounted displays, musical instrument design, design guidelines, performer perspective, musical expression, assessment methods, computer music
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Virtual Reality Musical Instruments: State of the Art, Design Principles, and Future Directions. https://app.cochat.ai/papers/10.1162/comj_a_00372
- The paper
- Serafin, S., Erkut, C., Kojs, J., Nilsson, N. C., & Nordahl, R. (2016). Virtual Reality Musical Instruments: State of the Art, Design Principles, and Future Directions. Computer Music Journal. https://doi.org/10.1162/comj_a_00372
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.