A Framework for the Evaluation of Digital Musical Instruments
The DOI matches its Crossref record; 44 of the 46 references it cites were found in a registry or an index.
Abstract
At the outset of a discussion of evaluating digital musical instruments, that is to say instruments whose sound generators are digital and separable though not necessarily separate from their control interfaces (Malloch, 2006), it is reasonable to ask what the term evaluation in this context really means. After all, there may be many perspectives from which to view the effectiveness or otherwise of the instruments we build. For most performers, performance on an instrument becomes a means of evaluating how well it functions in the context of live music making, and their measure of success is the response of the audience to their performance. Audiences evaluate performances on the basis of how engaged they feel they have been by what they have seen and heard. When questioned, they are likely to describe good performances as “exciting,” “skillful,” “musical.” Bad performances are “boring,” and those which are marred by technical malfunction are often dismissed out of hand. If performance is considered to be a valid means of evaluating a musical instrument, then it follows that, for the field of DMI design, a much broader definition of the term “evaluation” than that typically used in human-computer interaction (HCI) is required to reflect the fact that there are a number of stakeholders involved in the design and evaluation of DMIs. In addition to players and audiences, there are also composers, instrument builders, component manufacturers, and perhaps even customers, each of whom will have a different concept of what is meant by “evaluation.”
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Norman, D. (2016) · Vahlen eBooks · doi:10.15358/9783800648108 · Cited by 6,374
Visual perception of biological motion and a model for its analysis
Johansson, G. (1973) · Perception & Psychophysics · doi:10.3758/bf03212378 · Cited by 4,462
Sweetser (2005) · doi:10.1145/1077246.1077253 · Cited by 2,124 · cited 2×
“Sweetser and Wyeth (2005) have developed a model for evaluating enjoyment of game play in terms of “game flow.””
BackgroundMethod(1986) · North-Holland eBooks · Cited by 1,883
A Practical Guide to Usability Testing
Dumas, J. F., & Redish, J. C. (1993) · Cited by 1,335
Dedre Gentner, C. (1983) · Cited by 1,329
Getting a grip on tangible interaction: a framework on physical space and social interaction
Hornecker, E., & Buur, J. (2006) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124838 · Cited by 1,100
Information Processing and Human-Machine Interaction
Rasmussen, J. (1986) · Medical Entomology and Zoology · Cited by 907
Journal of Human Movement Studies
D., C. P. (1975) · The American Journal of Psychology · doi:10.2307/1421916 · Cited by 781
A taxonomy for and analysis of tangible interfaces
Fishkin, K. (2004) · Personal and Ubiquitous Computing · doi:10.1007/s00779-004-0297-4 · Cited by 422
Making sense of sensing systems: five questions for designers and researchers
Bellotti, V., Back, M., Edwards, W. K., Grinter, R. E., Henderson, A., & Lopes, C. (2002) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/503376.503450 · Cited by 405
Problems and Prospects for Intimate Musical Control of Computers
Wessel, D., & Wright, M. (2002) · Computer Music Journal · doi:10.1162/014892602320582945 · Cited by 387
HCI Models, Theories, and Frameworks
(2003) · Elsevier eBooks · doi:10.1016/b978-1-55860-808-5.x5000-x · Cited by 373
Visual Perception of Expressiveness in Musicians' Body Movements
Dahl, S., & Friberg, A. (2007) · Music Perception · doi:10.1525/mp.2007.24.5.433 · Cited by 311
About Face: The Essentials of User Interface Design
Cooper, A. (1995) · John Wiley & Sons, Inc. eBooks · Cited by 247
Evaluation of Input Devices for Musical Expression: Borrowing Tools from HCI
Wanderley, M. M., & Orio, N. (2002) · Computer Music Journal · doi:10.1162/014892602320582981 · Cited by 243
2001: Principles for Designing Computer Music Controllers
Cook, P. (2017) · Current Research in Systematic Musicology · doi:10.1007/978-3-319-47214-0_1 · Cited by 223
Theremin: ether music and espionage
(2001) · Choice Reviews Online · doi:10.5860/choice.38-3818 · Cited by 147
The 'E' in NIME: musical expression with new computer interfaces
(2006) · eScholarship (California Digital Library) · doi:10.5072/zenodo.205601 · Cited by 130
Motor Behavior Models for Human-Computer Interaction
MacKenzie, I. S. (2003) · Elsevier eBooks · doi:10.1016/b978-155860808-5/50003-4 · Cited by 126
2003: Contexts of Collaborative Musical Experiences
Blaine, T., & Fels, S. (2017) · Current research in systematic musicology · doi:10.1007/978-3-319-47214-0_6 · Cited by 119
Mathews, M. V. (1991) · Computer Music Journal · doi:10.2307/3681070 · Cited by 115
Hci Methodology For Evaluating Musical Controllers : A Case Study
Kiefer, C., Collins, N., & Fitzpatrick, G. (2008) · Figshare · doi:10.5281/zenodo.1179576 · Cited by 79
Using Contemporary Technology in Live Performance: The Dilemma of the Performer
Schloss, W. A. (2003) · Journal of New Music Research · doi:10.1076/jnmr.32.3.239.16866 · Cited by 76 · cited 3×
“Schloss (2003) , in particular, suggests that this disappearance of the relationship between cause and effect ultimately impacts upon the relationship between a performer and their audience., Schloss (2003) has noted that people attend musical performances in part to observe skilled players doing something they cannot…”
BackgroundRe-Designing Principles For Computer Music Controllers : A Case Study Of Squeezevox Maggie
Cook, P. R. (2009) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1177492 · Cited by 55
On Interface Expressivity: A Player-Based Study
Poepel, C. (2005) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1176801 · Cited by 52
(2006) · Cited by 51
Discourse Analysis Evaluation Method For Expressive Musical Interfaces
Stowell, D., Plumbley, M. D., & Bryan–Kinns, N. (2008) · University of the Arts London Research Online (University of the Arts London) · doi:10.5281/zenodo.1179630 · Cited by 49
Evaluating experience-focused HCI
Kaye, J. (2007) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1240866.1240877 · Cited by 48
A Manifold Interface for a High Dimensional Control Space
Choi, I., Bargar, R., & Goudeseune, C. M. A. (1995) · University of Michigan Library Repository · Cited by 36
Usability, playability, and long-term engagement in computer games
Febretti, A., & Garzotto, F. (2009) · CHI '09 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1520340.1520618 · Cited by 36
Thinking but not seeing: think-aloud for non-sighted users
Strain, P., Shaikh, A. D., & Boardman, R. (2007) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1240866.1240910 · Cited by 29
A Quantitative Evaluation Of The Differences Between Knobs And Sliders
Gelineck, S., & Serafin, S. (2009) · VBN Forskningsportal (Aalborg Universitet) · doi:10.5281/zenodo.1177548 · Cited by 26
Exploring two methods of usability testing: concurrent versus retrospective think-aloud protocols
van den Haak, M. J., & de Jong, M. D. T. (2003) · IEEE International Professional Communication Conference, 2003. IPCC 2003. Proceedings. · doi:10.1109/ipcc.2003.1245501 · Cited by 24
The Thummer Mapping Project (ThuMP)
Paine, G., Stevenson, I., & Pearce, A. (2007) · New Interfaces for Musical Expression · doi:10.1145/1279740.1279752 · Cited by 22
Digital Instruments and Players Part II: Diversity, Freedom and Control
Jordà, S. (2004) · University of Michigan Library Repository · Cited by 20
Nicolls, S. (2010) · Leonardo Music Journal · doi:10.1162/lmj_a_00012 · Cited by 20
Towards A Taxonomy Of Realtime Interfaces For Electronic Music Performance
Paine, G. (2010) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1177872 · Cited by 18
Audio Anecdotes: Tools, Tips, and Techniques for Digital Audio
Greenebaum, K., & Barzel, R. (2004) · A K Peters/CRC Press eBooks · doi:10.1201/9781439864029 · Cited by 16
Where Did It All Go Wrong ? A Model Of Error From The Spectator'S Perspective
Fyans, A. C., Gurevich, M., & Stapleton, P. (2009) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1177518 · Cited by 16
Proceedings of the 20th International Conference on Digital Audio Effects
Torin, A., Hamilton, B. J., Bilbao, S., & Newton, M. J. (2014) · Cited by 10
Creating a network of Integral Music Controllers
Knapp, R. B., & Cook, P. R. (2006) · Cited by 8
Sensory Chairs : A System For Biosignal Research And Performance
Coghlan, N., & Knapp, B. (2008) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1179512 · Cited by 4
Physical Synthesis of Bowed String Instruments
Serafin, S. (2007) · A K Peters/CRC Press eBooks · doi:10.1201/9781439864869-6 · Cited by 3
Research works printed but not linked: 11
- Blaine T., 2003, Proceedings of the International Conference on New Interfaces for Musical Expression (NIME). Montreal, Canada: McGill University, 129
- Choi I., 1995, Proceedings of the International Computer Music Conference. San Francisco, California: International Computer Music Association, 181
- Coghlan N., 2008, Proceedings of the International Conference on New Interfaces for Musical Expression (NIME). Genova: Casa Paganini, 233
- Cook P., 2009, Proceedings of the International Conference on New Interfaces for Musical Expression (NIME). Pittsburgh, Pennsylvania: Carnegie Mellon University, 218
- Fyans A. C., 2009, Proceedings of the International Conference on New Interfaces for Musical Expression (NIME). Pittsburgh, Pennsylvania: Carnegie Mellon University, 172
- Gelineck S., 2009, Proceedings of the International Conference on New Interfaces for Musical Expression (NIME). Pittsburgh, Pennsylvania: Carnegie Mellon University, 13
- Jorda S., 2004, Proceedings of the International Computer Music Conference. San Francisco, California: International Computer Music Association, 706
- Kiefer C., 2008, Proceedings of the International Conference on New Interfaces for Musical Expression (NIME). Genova: Casa Paganini, 87
- Paine G., 2010, Proceedings of the International Conference on New Interfaces for Musical Expression (NIME). Sydney: University of Technology Sydney, 436
- Poepel C., 2005, Proceedings of the International Conference on New Interfaces for Musical Expression (NIME). Vancouver: University of British Columbia, 228
- Stowell D., 2008, Proceedings of the International Conference on New Interfaces for Musical Expression (NIME). Genova: Casa Paganini, 81
Cited by
Accessible Digital Musical Instruments—A Review of Musical Interfaces in Inclusive Music Practice
Frid, E. (2019) · Multimodal Technologies and Interaction · doi:10.3390/mti3030057 · Cited by 177
Virtual Reality Musical Instruments: State of the Art, Design Principles, and Future Directions
Serafin, S., Erkut, C., Kojs, J., Nilsson, N. C., & Nordahl, R. (2016) · Computer Music Journal · doi:10.1162/comj_a_00372 · Cited by 139
“Even then, several aspects have to be assessed, from mere task performance, as carried out by Wanderley and Orio (2002), to the playability, user experience, and emotions, as outlined by O’Modhrain (2011).”
BackgroundMind, Body, World: Foundations of Cognitive Science
Dawson, M. R. W. (2013) · Athabasca University Press eBooks · doi:10.15215/aupress/9781927356173.01 · Cited by 126
Design Considerations for Real-Time Collaboration with Creative Artificial Intelligence
McCormack, J., Hutchings, P., Gifford, T., Yee-King, M., Llano, M. T., & D’inverno, M. (2020) · Organised Sound · doi:10.1017/s1355771819000451 · Cited by 101
“…methodological consideration that has received much support from researchers is the use of subjective aesthetic assessment by ‘ domain insiders ’ as an instrument for evaluating improvisation (Stowell, Robertson, Bryan-Kimms and Plumbley 2009; O ’ Modhrain 2011; Linson, Dobbyn and Laney 2012).”
Smart Musical Instruments: Vision, Design Principles, and Future Directions
Turchet, L. (2019) · IEEE Access · doi:10.1109/access.2018.2876891 · Cited by 94
“New DMIs may succeed where they contribute to expanding the creative possibilities of the player [77], [82].”
Background
Metadata
- Authors
- O'Modhrain, S.
- Affiliations
- Sonic Arts Research Centre, Queen's University Belfast, University Road, Belfast, BT7 1NN, Northern Ireland.
- Venue
- Computer Music Journal
- Publisher
- MIT Press - Journals
- Published
- 2011-03
- Volume
- 35
- Issue
- 1
- Pages
- 28-42
- ISSN
- 0148-9267 (ISSN-L)
- 1531-5169
- OpenAlex
- W2044170912
- Semantic Scholar
- 653292b4bad6e54bd5fecccd02bf6aa1b3a1f983
- Cited by
- 130
- References
- 46
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 6 · 2023: 11 · 2022: 7 · 2021: 7 · 2020: 12 · 2019: 8 · 2018: 11 · 2017: 10 · 2016: 11 · 2015: 6 · 2014: 11 · 2013: 8 · 2012: 11
- FWCI
- 11.37
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Computer Music Journal (publishedVersion)
- Project Muse (Johns Hopkins University) (submittedVersion)
- Research Portal (Queen's University Belfast) (submittedVersion)
- Topics
- Social Sciences › Arts and Humanities › Music › Music Technology and Sound Studies
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- digital musical instruments, instrument evaluation, performance evaluation, stakeholder perspectives, audience engagement, human-computer interaction evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.