Download a pdf version HERE

Contents

1. Abstract
2. Keywords
3. Introduction
4. Main Concepts
5. Social aspects
6. Prototype Overview
7. Proposal
8. Conclusion and Next Steps
9. References

1. Abstract

This thesis presents an interactive system that allows people with no previous musical training to participate in a collaborative experience based on sound. The system allows novices to understand the social and psychological implications of making music as a group, with the aim of making collaborative musical experiences more approachable to them, blurring the distinction between music performers and music listeners. The experience encourages communication and collaboration through playful interactions to ultimately experience a state of group flow.

The goals of this thesis are:

• Investigating the social interactions in a novice sound environment.
• Make use of technology to facilitate and drive musical experiences for nonmusicians, encouraging communication, collaboration and creative engagement with sound.
• Provide a space where, with a small set of rules, nonmusicians can collaboratively explore, improvise and have fun with sound in a similar fashion as trained musicians would.

2. Keywords

Collaboration, music, play, state of flow, joy.

 

3. Introduction

Historically, music has served as a cohesive instrument for communication, defining and keeping communities together, for celebration and for mourning. Music is among of one the most ancient and ubiquitous human activities. Archaeologists have found that some of the earliest tools created by humans were musical instruments[1], this supports the argument that music is one of the core elements of social organization and one of the things that defines us as humans. However, contemporary music making has become an activity mostly reserved for some people: musicians. In his book This is Your Brain on Music, Daniel J. Levitin explains this distinction by saying:

“Only relatively recently in our own culture, five hundred or so ago, did a distinction arise that cut society in two, forming separate classes of music performers and music listeners. Throughout most of the world and for most of human history, music making was as natural an activity as breathing and walking, and everyone participated. Concert halls, dedicated to the performance of music, arose only in the last several centuries” (Levitin, 6).

I don’t consider myself a virtuoso instrumentalist, or even a musician. Musical notation is as foreign as Russian language is to me, and I don’t really understand musical scales. Yet, I’ve been making music for more than fifteen years, and I’ve managed to play in some bands. My sense is that most of musicians don’t really care about the technical aspects of music, they make music because they like it, because it is a rewarding and pleasurable activity and playing with someone else just amplifies those feelings. So I wonder: why the distinction between performers and listeners is so prevalent, when making music is one of the things that fundamentally defines us as humans?

 

Statements as “I don’t know how to play an instrument, therefore I can’t make music“ or “ I can’t play music with others, I’m not a musician! “ are very common between nonmusicians. People are inherently afraid of playing an instrument, and much more of doing so in front of others. The musical experience becomes then something unreachable for most people, and only reserved for an elite.
So, what role can a Design and Technology thesis can play on this issue? How can technology facilitate musical experiences for nonmusicians?  These are the questions that drive my exploration and that I will try to answer through my thesis project.

 

This thesis investigates the social interactions in collaborative sound environments. The aim of the project that illustrates this thesis is to provide a structure where participants are encouraged to achieve a loose state of mind and openness towards sound exploration. This state of mind sets the basis for the development of collaborative interactions, enhancing the social cohesion between participants and ultimately leading to a state of group flow. Setting up small sets of rules for the interaction, the structure should allow emergent play and musical expression, as means to achieve the desired social experience.

 

Design questions

• How to engage novices in collaborative sound experiences?
• How can technology facilitate the music and social experience?
• How can an interactive system encourage collaboration and allow musical expression?

 

4. Main Concepts

My research for this thesis was focused on the dynamics of engagement, as a way to understand how people can be encouraged to participate in a collaborative experience around sound. Three concepts stand out as the pillars where my project will stand: play, flow and entry-level.

Play
In contrast to adults, most of young children experience music making in a regular basis. In their music class at the elementary school, children are encouraged to play instruments and explore music together through play. In their minds there is no such thing as novices and experts, performers or listeners. For them, making music is something natural, and something extremely fun to do with others. This openness towards musical experiences is lost as we grow old, becoming something that only musicians do.

My thesis proposes that playful interactions are an effective way of facilitating musical experiences for novices. This is supported by a series of prototype tests that will be explained later on this text. The main idea behind this is making the participants feel that they are “playing” instead of “performing”, so the idea of “making music in group” becomes more plausible for nonmusicians.

 

Flow
Trained musicians know that making and performing music with other players leads–most of the times–to a unique state of mind: you are not focused on yourself, you are focused on the collective action, on the others, on music. Your self-consciousness is reduced and the perception of the synergy generated as a group is enhanced. There’s a pleasurable and rewarding feeling about the idea of creating something together, becoming a ritual that enhances the sense of communitas [2].
Mihaly Csikszentmihalyi names this as Flow [3], a mental state of complete energized focus on a certain activity. Making music is one of the activities where flow is easily experienced. In a TED talk, Csikszentmihalyi quotes a music composer he interviewed:

“You are in an ecstatic state to such a point that you feel as though you almost don’t exist. I have experienced this time and again. My hand seems devoid of myself, and I have nothing to do with what is happening. I just sit there watching it in a state of awe and wonderment. And [the music] just flows out of itself.”

We experience flow in a variety of activities in our everyday lives: when we play sports, solve a math problem or make a sculpture. The dynamics of flow are related to the balance between the challenge that a certain activity imposes and the skills associated to those challenges. An activity becomes boring when the challenges are low for our given skills, in contrast, it leads to anxiety when the challenge surpasses our skills. That’s why different people experience flow in different activities.
Flow is a necessary condition for engagement. My thesis proposes that facilitating flow states, the experience becomes more rewarding and appealing for nonmusicians, therefore encouraging musical exploration.

 

Entry Level
The physical attributes of instruments gives them their particular sound, a piano sounds like a piano because it has a large resonant box with two hundred and thirty strings inside that can be excited by small hammers when pressing a keyboard. However, this same physical attributes impose learning challenges for playing the instrument. A violin for instance, can take several years to be mastered until it is used in its maximum expressive potential. This learning curve is in part why nonmusicians are usually afraid to play an instrument.
However, the current state of digital technology and sound synthesis makes music no longer attached to physical attributes, therefore allowing very arbitrary mappings. By this way, we can start designing musical interfaces that are more approachable by novice users, facilitating the musical experience and lowering the entry-level.

 

5. Social aspects

Trying to understand what are the conditions that facilitate engagement, I looked into social interactions that imply both a behavioral change and the presence of flow. Native American rituals around the drum, for instance, are cohesive social experiences where music acts as an instrument for reinforcing the ties within the community. In these rituals, participants sit around a drum, and with a drum stick they hit the drum in a constant and hypnotic pace. The beauty of this ritual relies on the fact that an extremely simple gesture to operate a simple interface is all what is needed to experience flow and make the ritual work.
Part of my research involved understanding how certain circumstances can change people’s behavior, making them become completely open towards a given activity. I’ve seen that behavioral change is profoundly influenced by the nature of the context where the activity takes place. This behavioral change takes place as a response to the special rules that exist only inside that particular space. Salen and Zimmermann describe this as the Magic Circle[4], a parallel space of reality where the world is inhabited only by the participants of the activity:

“…Within the magic circle, special meanings accrue and cluster around objects and behaviors. In effect, a new reality is created, defined by the rules of the game and inhabited by its players.”

For example, the karaoke bars and booths–so popular in Japan–are spaces that meet those conditions. Karaoke spaces are usually used by groups of friends looking for an entertaining experience. Within this space, people behave in a completely different way as they would do outside. Inside, they are not afraid of singing, and feel no shame about themselves. In a similar fashion, Photo-Booths are also spaces with their own particular rules. It is interesting that although all what these booths do is nothing more than taking a set of pictures, there is something very unique about entering a very tight box, making to people share their personal space with others to behave as children.

 

6. Prototype Overview

Methodology
Through a process of iterative prototypes, I performed user tests with the aim to get some answers for three main questions: How people respond to different interface configurations? How the interface’s affordances can shape the experience? What are the conditions for a system to facilitate flow and joy?
The methodology was based on building six prototypes and testing them with users to observe and analyze their reactions. On every iteration, different mapping configurations were applied, progressively increasing in complexity and incorporating the findings from the previous versions.

 

Prototype One: Collaborative Rhythm

Three players are placed around a midi device equipped with buttons that trigger different sounds. Each player operates one portion (up to two buttons) of the interface.
The experience is divided into two sections that differ in complexity: on the first, they operate only one button and its corresponding sound. On the second, each player plays two buttons, consequently taking care of two sounds.
The goal is to collectively create a rhythm where each participant plays only one sound at the time, to make it clear, it is basically like dividing the role of a drummer within three participants where one plays the snare-drum, the second plays the kick-drum and the third one the hi-hat.
Participants can play one sound at a time and only in sequence, after the previous player’s turn.

More info HERE

 

Prototype Two and Three: Body Gestures

Using a pair of earphones, the user stands in front of a camera that is connected to a computer. On the screen, the user can see itself. The core of the interface is a program written in openFrameworks, that using camera vision techniques is capable of understanding wether the user extends the arms to the right, left or up. Each one of these gestures trigger different sounds that the user can hear through the earphones. The triggering is supported by visual feedback: a red dot appears indicating when and which sound is on. No specific directions are given besides of staying in sight of the camera.

More info HERE

Prototype Four: Sound Installation in Public Space

A computational sound interface is placed in a highly transited zone, where people usually walk by without stopping. The interface is relatively hidden from people’s sight so participants can’t notice its presence until they are close enough. Besides of the placement of a pair of speakers and a small webcam, there are no visual cues that indicate the presence of the installation or its operation.
The interface uses a camera to sense the movement and proximity of the people who walk by–or enter– the visible space of the camera. Based on the amount of movement and proximity, the interface outputs different sonic feedbacks.
Once the users are in the camera’s sight, a sound becomes louder or lower depending on how much the users move.  If the participant freezes for instance, the sound becomes inaudible. In contrast, if the participant moves, the sound becomes louder. In addition, a second input that senses proximity to the camera makes a second sound with different timbre and frequency qualities become louder or lower.

More info HERE

Prototype Five and Six: Pyramid and Pendulum

I developed two prototypes using a structured system of three tracking points placed in a triangular configuration on tangible controllers. When moving the controllers in space, the position of each one of the points–and their distance–modify different sonic parameters such as filters or triggering a particular sound. The position of the tracking points is captured using camera vision techniques, and processed using a program written in openFrameworks that generates midi messages based on the position variations of the points. The sound is generated using Ableton Live, a sound software that has the capacity of receiving and mapping midi messages into different parameters. Visual and sonic feedback is provided using a large computer screen.
The two upper points control the filter of a sound. The lower point triggers a sound when it crosses a virtual line drawn in the center between the two upper points. Pyramid, the first iteration, is a single-user controller shaped as an inverted equilateral triangle, and Pendulum, is a controller operated by two participants that make the third point move in a pendular movement using two rods connected with a rope, from where the tracking point hangs from a weight to allow a pendular movement.

More info HERE

7. Proposal

Abstract

This thesis presents an interactive system that allows people with no previous musical training to participate in a collaborative experience based on sound. The system allows novices to understand the social and psychological implications of making music as a group, with the aim of making collaborative musical experiences more approachable to them, blurring the distinction between music performers and music listeners. The experience encourages communication and collaboration through playful interactions to ultimately experience a state of group flow, a necessary condition for engagement.

Design Questions

How to engage novices in a collaborative sound experience?
How can a structured system facilitate engagement and foster collaborative musical expression?

Goals

The goals of this project are:

• Make use of technology to facilitate and drive musical experiences for nonmusicians, encouraging communication, collaboration and creative engagement with sound.
• Provide a space where, with a small set of rules, nonmusicians can collaboratively explore, improvise and enjoy with sound in a similar fashion as trained musicians would.
• Create a scalable structural system that, maintaining the same basic principles, allows scaling to different number of participants.
• Creating a system that encourages personal and collective exploration and allows emergent play.

 

Concept and Social model

The project relies on a conceptual model where each participant can explore, at the same time, the individual and collective space of interaction with sound. This coexistence of both spaces is reflected on the configuration of the controller, that by its physical affordances requires to be operated in collaboration.

 

 

Structure and Interaction

The interface consists on a series of tangible geometrical structures built with rope that are operated by the participants in collaboration. These structures act as controllers of a computational system that processes and generates sound. The basic idea is trying to maintain the tension of the rope.  To do this, users need to communicate, coordinate their movements and make collaborative decisions in order to move while maintaining the shape of the structure.

The gestural capture strategy is based on a series of tracking points attached to the physical controllers, that when moved in space, modify the different parameters of the sonic output. Using camera vision techniques, the position of the tracking points in space can be mapped arbitrarily to different parameters of sound such as pitch, filters or notes.

 

Mapping

The mapping system is largely based on the Pyramid and Pendulum’s system where the basic principle is tracking the position of points in a coordinate system to control sound parameters. This allows, for instance, mapping the position of the structure in a room equipped with a camera on the ceiling, making participants to coordinate to move the structure around the room.

 

Scale

The proposal involves exploring different scales of participation, where groups constituted by different number of players can interact. For this, the plan is to build a series of structures that allow from two to five participants.

 

8. Conclusion and Next Steps

This has been an exciting and intense process. By doing research I’ve been able to create a consistent discourse that supports the proposed project. My plan for next semester is to build and test the system using different metrics: scale, context, level of physicality and different sound mappings.

 

9. References

[1] Levitin, Daniel. This is your brain on music : the science of human obsession. New York  N.Y.: Dutton, 2006

[2] Turner, Victor. The ritual process : structure and anti-structure. New York: Aldine de Gruyter, 1995.

[3] Csikszentmihalyi, Mihaly. Flow: The Psychology of Optimal Experience. New York: Harper & Row, 1990.

[4] Salen, Katie and Zimmerman, Eric. Rules of play: Game Design Fundamentals. London: The MIT Press, 2004.