Abstract

With the popularization of mobile computing and software-distribution platforms like the Apple Store and Google Play, a plethora of remarkable software-based musical interfaces with dynamic visual feedback are being designed today. Recent examples such as the Loopy, the Borderlands, the Ocarina illustrate that powerful visual interfaces are capable of captivating amateur and expert musicians alike by making music manipulation more direct and straightforward. However, while successful examples are available, effectively designing such direct interfaces remains challenging, typically proceeding “as more art than science”. For example, unlike acoustic instruments, sound production is physically decoupled from input controls, yielding infinite possibilities for mapping. Moreover, increasing ease-of-use for novices can lead to toy applications that disinterest experts. Finally, analyzing successful crafts remain unusual, and little music interface research is being carried to better understand these interfaces. This thesis attempts to understand these musical interfaces better, aiming at providing empirical evidence to inform their design and evaluation. Borrowing from human-computer interaction research, I hypothesize that one effective strategy concerns adopting an interaction model known as direct manipulation–characterized by continuous visual representation of the object of interest, physical actions to manipulate the visual representations, rapid visual feedback, and incremental and easily reversible actions. To investigate this hypothesis, I report exploratory case studies where direct manipulation is designed and evaluated in three specific contexts of music tools: a) live looping tools; b) tools for creating interactive artistic installations; and c) music authoring tools. As a result, I introduce three novel software music tools: Voice Reaping Machine; ZenStates; and StateSynth. Finally, I present a set of interface design strategies and evaluation approaches for effectively designing direct manipulation in musical interfaces based on visuals.