PhD Final Seminar

  • CSIT (Building 108) The Australian National University

Apps, Agents, and Improvisation: Studying Ensemble Interaction with Touch-Screen Digital Musical Instruments


This project is about performing with new digital musical instruments (DMIs) on touch-screen devices that are designed for ensemble performance. While computer music has advanced to the point where a huge variety of digital instruments are common in educational, recreational, and professional music-making, these instruments rarely seek to enhance the ensemble context in which they can be used. In this research, new ensemble-focussed instruments have been designed and deployed in an ongoing artistic practice, these instruments have been evaluated to find out whether and how they affect the ensembles and music that is made with them.

A series of improvised performances with prototype apps exposed a vocabulary of continuous performative touch-gestures. A system for tracking these collaborative performances in real time was developed using tools from machine learning. This tracking system is posed as an intelligent agent that can continually analyse the performers and trigger changes in their user interfaces at appropriate moments. Two formal studies examine performers' responses to several app-agent designs using this system.

This project has examined ensemble DMI performance in unprecedented scope and detail - over 150 interaction sessions have been recorded. Informed by the results of lab and field studies using quantitative and qualitative methods, four generations of ensemble-focussed interface have been developed. The results assure us that the intelligent agent interaction produces performances that are longer, with improved perceptions of musical structure, group interaction, enjoyment and overall quality.