ABSTRACT
This paper explores the role of materiality in Digital Musical Instruments and questions the influence of tacit understand- ings of sensor technology. We suggest that a tacit knowledge of the ‘sensor layer’ brings with it definitions, understandings and expectations that forge and guide our approach to interaction. We argue that the influence of technology starts before a sound is made, and comes from not only intuition of mate- rial properties, but also received notions of what technology can and should do. On encountering an instrument with obvious sensors, a potential performer will attempt to predict what the sensors do and what the designer intends for them to do, becoming influenced by a machine centered understanding of interaction and not a solely material centred one. The paper presents an observational study of interac- tion using non-functional prototype instruments designed to explore fundamental ideas and understandings of instrumental interaction in the digital realm. We will show that this understanding influences both gestural language and ability to characterise an expected sonic/musical response.
DISCUSSION
Materiality in DMI design
It is clear from the study that materiality plays a huge role in expectation of both gestural interaction and sonic re- sponse in DMIs. The wood and rubber prototypes evoked categorically different responses, with one seen as percussive and the other pressure based. This understanding influenced not only the choice of gesture used, but assumptions of what they would sound like. In many cases participants drew an explicit analogy between the natural sound of the material and the kind of sound the instrument should make. This conclusion was strengthened by the fact that the sensor prototype was seen not to have a natural sound at all. The difference in response between the wood, rubber and sensor based instruments show that our assessment of po- tential gesture is guided by the playing surface, and beyond this our expectation of what the instrument does and how it sounds is fundamentally linked to material considerations.
A machine-centered approach to DMIs
The outcome of the study highlights a limitation in rela- tion to explicit sensor technology, which we attribute to a ‘machine centered’ approach to instrumental interaction. Participants exhibited a very different understanding of the sensor prototype, becoming preoccupied with the kind of spatial representation they assumed the digital sensor would produce. The prototype was seen as a controller and not an instrument, and promoted XY-based thinking adopted from smartphones rather than acoustic instruments. This under- standing was also evident in the actions of a few participants who assumed the prototypes might contain accelerometers or gyroscopes, which subsequently influenced their gestures. Notions of fragility, lack of feedback, and a two dimensional pattern of interaction underpinned many responses. The instrumental gestures used with the wood and rubber pro- totypes gave way to notions of basic control, mappings and triggering; losing sight of control intimacy, gestural rich- ness, and sonic nuance, as desired by many writers and researchers in the field.
Non-functional prototypes
The use of non-functional prototypes as an experimental methodology has proven to be fruitful. The ability to sep- arate the expectations of a technology from the reality of using a technology has enabled an assessment of the tacit cultural elements that underpin our understandings and in- fluence our subsequent actions. This paper shows the value of the approach in the context of DMI and NIME design. The use of a non-functional prototype design stage in NIME development could afford an indication of what kinds of per- formance techniques are likely to emerge, which could be reinforced of deliberately thwarted in the development of elements such as sensor choice, placement and mappings.
Made with Mobirise
