Real Gestures, Virtual Environments

Real Gestures, Virtual Environments
Susan Kozel and Kirk Woolford
Published in On Line: Performance Research, Vol. 4: No. 2, Summer 1999. pp. 61-64.

During the creation of contours, Woolford and Kozel participated in the 1998 ESPIRIT workshop “Real Gestures, Virtual Environments” lead by Sally Jane Norman. During the last day of their work at the Zentrum für Kunst und Medientechnologie (ZKM), Woolford and Kozel performed an experiment allowing participants to “puppet” their own movement. Participants in the experiment captured a movement sequence, using an electro-magnetic motion capture system (Polhemus Ultra-trak filtered through Maya) then replayed this movement through a program displaying a mix of their current and past positions based on their movement.

Because of time and equipment limitations, the puppet’s bodies were reduced to two simple curves running down the left and right sides of the body with a cross-mark for the head. Even at this level of extreme abstraction, all the workshop participants could easily recognize who’s body and movement they were watching, what day they were captured, and often, what time of day based on the posture and dynamism of the movement.

Two projects by Mesh, Contours and Figments, have been commissioned to première in the UK during 1999 and then to tour. The movement vocabulary, theoretical vocabulary and software have been created together by Kozel and Woolford over the past year. They are both about a confrontation between virtual and physical bodies and spaces. Contours is based around computer vision and movement tracking software, and Figments uses a combination of ultrasound and magnetic motion capture systems. Kirk Woolford has written the software for both pieces, Susan Kozel and Ruth Gibson are the performers.

Susan Kozel: ‘being in the system’
The starting, middle and end points of the work revolve around being in the systems. Whether the system is motion capture, motion tracking or videoconferencing, the only way (as a performer and deviser) I can make sense of what is happening and begin to build a physical and philosophical vocabulary is by spending hours and hours ‘wired up,’ metaphorically, physically and visually.

What emerges is often unexpected. Research into Figments at the ZKM made me realise that dancing with a virtual body is like dancing with another being. (2) The virtual body (middle figure in the first series of images) was a modulation between a pre-recorded sequence and the data generated from my real-time movement in the magnetic motion capture system. This ‘third figure’ was synthetic but took on a movement quality and even an emotional character of its own. My movement dialogue with the third figure became one of choreography and control. In some ways I was spooked by it.

‘Control’ is another way of addressing how the performing body inhabits the systems. The software for part of Contours (second series of images) creates a gentle and fluid environment, it invites a languid and sensual engagement. It is, in the words of the audience, like swimming in light. For another part, the software throws out an aggressively searching grid and, instead of swimming, I was drowning. Until I found my place within that system, until I learnt how to inhabit it through movement initiation and response, it seemed to take over the entire surface of my body in a non-specific yet frenetic way.

Controlling the grid in Contours, or the third figure in Figments, comes about through developing a new relationship with my own body -- for I am the interface in a unknown system. But the control needs to be established so that it can in turn be relinquished. The aim is not to control like a videogame, but to transform the relation between performer and software into a dialogue. This dialogue happens on many visual, formal and physical levels at once: speed, 3-d planes, bodily parts and patterns in space.

Kirk Woolford: ‘seeing through the system’
While discussing Mesh’s work, a colleague recently remarked that he didn’t believe our work was influenced by photography. This caught me off-guard because of my background and because our systems draw so heavily from computer vision and image processing. Whether we’re involved photography, cinematography, or any-other-ography is an issue for another article. One thing is certain, these system do see -- and we use their viewpoints to redirect our own.

Figments uses a different form of technology, but a similar concept. Contours watches the performers through cameras, but Figments places sensors on various sections of the body and hears where they are in the space. Figments uses the movements of the live performer to mix and modulate the movements of the virtual performer. In order to do this, we must know what is moving where, and how. After all the filtering and processing of the data from the motion-capture suits, we’re left with the same questions. How do we represent all the complexities we perceive as a simple movement and how can we use the technology to redirect both the performer’s and audience’s understanding of the movement.