An immersive piece  addressing climate change and loss of biodiversity in the United States through a crime-fiction narrative exploring the disappearance of fireflies. Photuris asks audiences to reflect upon the cultural, ecological, and financial value of the firefly.

• Fireflies are a part of our biodiversity heritage and are iconic insects that have been the subject of much investigation in the sciences, an inspiration in the arts and a part of local cultures, folklores and traditions because of their ability to produce light.

• Fireflies have been a source of ecotourism revenue for many communities in different parts of the world and have the potential to bring similar benefits to other local communities. Fireflies and their natural habitats also enhance quality of life and contribute to economies through the promotion of aesthetically pleasing landscapes that have greater appeal.

• Fireflies are bio-indicators of the health of the environment and are declining across the world as a result of degradation and loss of suitable habitat, pollution of river and water systems, increased use of pesticides in agro-ecosystems, non-regulated commercial harvesting and increased ecological light pollution in areas of human habitation.

While observing firefly behavior, several naturalists have noted that females of the genus Photuris are carnrivorous. Many have discovered this by trying to keep groups of fireflies alive overnight in the same container. In the morning one usually finds one Photuris female and bits and pieces of all the rest.

More detailed information about the project can be found on the project website at http://resounding.place/

The vRSP network explores new forms of immersive performance experience in sites of historical and cultural import. It will start by examining new works by composer Michael Price who has been exploring the use of 360 video and audio to document performances in National Trust properties. The research network will examine how these performances have been recorded and converted to 360 videos, and offer new solutions using ongoing research from the University of Surrey’s Institute of Sound Recording, Center for Vision, Speech and Signal Processing, 5G Innovation Centre, and Digital Media Arts programmes.

Many spatial experiences are not well represented through current 360 video and immersive technologies. Despite Ambisonics being adopted as the de-facto soundfield capture and manipulation standard by Facebook and other 3D-audio content generation platforms, current 360 video experiences lack spatial audio detail and the ability to explore the soundfield convincingly. This leads to a considerably reduced immersive experience. While equipment manufacturers currently focus on increased image quality through higher pixel counts, there is currently a gap in both documentation and content of high-quality spatial audio which has been captured outside of game-engine based experiences.

The research network will explore how new forms of immersive experience can be created to allow future audiences to move through the space during the performance and explore the relationship between the performance and location. The network will also examine how extra layers or “maps” of the site can be added to allow the audience to explore both the history and significance of the site as well as the process of creating the work. Finally, the vRSP network will explore how an immersive experience can go beyond what is possible in a live performance to allow the experience to respond to the presence of the audience.

vRSP is developing outputs which are exemplars of a wholly new genre of art work, designed for virtual space but based on notable and rare historic places for which music will be written to integrate musical and social memory, and exploit unique architectural and acoustic environments. Live performances of newly composed works will be captured and disseminated to viewers and listeners as VR content through the application of a series of novel production techniques. These will enhance and address specific difficulties presented by current technologies, as described below, by applying interdisciplinary knowledge combining research from human perception, neuroscience, musical composition, animation, visual surface-mapping techniques and visual signal processing, stereoscopic 360 degree video and three- dimensional audio recording and reproduction methods. The network draws together experienced practitioners from all fields to inform the creation of a new work that will portray the intrinsic qualities of the historic performance space through site specific live performance and a combination of contemporary video- and animation-based techniques for VR generation.

The core project team includes Michael Price, one of the UK's most sought-after composers, known for Emmy-winning compositions for BBC's Sherlock and Unforgotten as well as work on films including Peter Jackson's The Lord of the Rings trilogy, Richard Curtis' Love Actually, Bridget Jones: The Edge of Reason and Alfonso Cuaron's Children of Men. Alongside his Film/TV work, and signed with the label Erased Tapes Records, Price is releasing a new album recorded on six locations across the UK, ranging from a Tudor mansion to tunnels within the White Cliffs of Dover. Each piece was recorded onsite and composed in response to the site and its history. After enquiring with industry colleagues about methods of recording and reproducing immersive audio, Price began consulting directly with equipment manufacturers in search of solutions, becoming an associate member of the BAFTA-VR Advisory Group. vRSP will allow Price to work together with University of Surrey academics Kirk Woolford and Tony Myatt. Woolford is an expert on interactive media content creation with more than 150 interactive works created, 30 years' experience developing immersive performances, including 20 years of VR performances, and 8 years' work with AR. Myatt is the head of Music and Media at the University of Surrey, founding director of the University of York's Music Research Centre, one of the foremost authorities on spatial audio. All three core partners are active creative practitioners and perform their works world-wide.

Jinn-Ginnaye is a practice-based research project developed in parallel with the live Choreomusical work, Jinn. Impetus for Jinn-Ginnaye came from experiences of censorship when the Interational Society of Electronic Arts (ISEA) conference was held in Dubai in 2015, and all images of womens' bodies had to be removed from public presentations. Jinn-Ginnaye asks how the constraints of Islamic culture can lead to new forms of creation and expression.

Jinn-Ginnaye is an exploration of movement in place. It is a collection of dance pieces exploring issues of bringing western dance performance to the United Arab Emirates, where local modesty laws influence how women can be shown in public. The pieces use video compositing, motion capture, and Virtual Reality techniques to remove the body of the dancer, but leave behind the dance, and the traces of the desert in which it was created.

The Jinn project was initiated by Carlos Guedes is interested in exploring the unique sounds of movement in the desert surrounding Abu Dhabi. He had been experimenting with methods of capturing the sound of wind across the dunes when he noticed how various forms and densities of sand created vary different sounds as he walked through them. In order to explore this further, Guedes invited a number of dancers to go out into the dunes with him, and they experimented with the best forms of movement to create sound. While I was in Abu Dhabi to make a site-specific motion capture piece for the ISEA 2014 conference, Carlos invited me to take one of my inertial motion capture suits into the desert so we could capture both the sound and the movement of a dancer in the desert.

United Arab Emirate modesty laws place restrictions on depictions of the human body in public. I decided to use the UAE restrictions as a creative tool, and developed software to create a “sand dancer” which would not offend our hosts in the UAE, but would tap into local legends of Jinn, and the older Ginnaye linked to the term Genius Loci, or spirit of a place. The sand dancer needed to reform and scatter – taking human form only long enough to be recognized, then blowing like sand in the desert wind.

Over the course of 2 years, we built up a collection of audio, video and movement recordings together with a 360 spherical images of the capture locations. I developed custom software using the Unity3D game engine, Google Cardboard, Samsung Gear VR and the Oculus Rift DK2 to place these sand dancers in the middle of the middle of the Rub Al Khali desert. Through these VR tools, we were able to and to allow audiences to instantly travel to one of the most inhospitable locations on the planet, and meet a representation of the spirit of the place.

During performances of Jinn in Abu Dhabi, audiences saw a live dancer perform in front of a video projection filmed in the Rub Al Khali desert. After several performances, we distributed Samsung Gear VRs, and iPhones with Google Cardboard viewers. We put the dancer into an inertial motion capture suits and performed a sand dance where the dancer’s movements were transmitted to the VR viewers and the audience could see her projected in 360 degree stereo into the remote location. The audience reported that the VR viewers gave the performance a more immediate context. They felt a clearer, stronger, link to the original location after viewing it in VR, but none of the audience members watched the entire performance in the immersive environment. They all chose to take off the 3D viewers, and watch the live dancer with in front of them. Two audience member reported that the sand dancers felt like a gimmick when viewed through the VR display, but viewing the desert location through the VR goggles changed their experience of the whole performance. They felt a greater degree of presence. Their experience of the performance was enhanced by this new form of photography, but they felt the presence of the live performer was stronger when she stood on the stage in front of them.

An introduction to Virtual/Augmented Reality wellness apps presenting "natural" environments prepared for the Valuing Nature annual conference 2019 at the Royal Society, London

The video presents an overview of meditation apps, which attempt to place their users in natural environments – assuming that "natural" experiences must be calming. It includes screen recording from within a number of VR wellness apps.

https://valuing-nature.net/ValNat19/overview

https://valuing-nature.net/cinema

Jinn — Jinn is a choreomusical work, set in the desert of Rub’ al Khali, exploring dance and music in the United Arab Emirates, and exchanges between Islamic and Western cultures.

On Wednesday 10 October 2018, Kirk Woolford, Reader for Digital Media Arts at Surrey University, and Carlos Guedes, Associate Arts Professor NYU, premiered their new piece, “Jinn” at NYUAD Arts Centre’s, in Abu Dhabi.

Using live motion capture, virtual puppetry, live musicians and a multitude of sand, Jinn is a choreomusical work exploring the presentation of Western performance in Arabic cultures. The Arabic term ‘jinn’ refers invisible, sentient beings, of scorching winds and a smokeless fire. Before Islam, they were worshiped as spiritual protectors not only in the Arabian Peninsula, but also in neighbouring areas. For this performance piece, the collaboration draws inspiration from the sound of movement in the desert and images of Jinn in Arab culture.

NYU’s agreement with the Emirate of Abu Dhabi to create NYU Abu Dhabi is the outcome of a shared understanding of the essential roles and challenges of higher education in the 21st Century and a common belief in the value of a liberal arts education.

Through Jinn, we wanted to present a wider Western culture to Emiratis. We’ve adressed constraints of showing the female form in a Muslim culture, by ultimately creating a dance piece without a dancer: removing the physical body, and leaving just the sound and trace of the movement, and transforming the performers into creatures of sand and fire.

Jinn is set in an area of the Abu Dhabi desert known as the empty quarter. It explores the sound of human movement in this environment, as captured by video, audio, and motion recording devices. The piece combines dance, live music performance with multichannel sound diffusion, motion capture and computer-generated graphics.

Guedes and Woolford have developed creative works addressing the perception of human bodily motion since 2003,in pieces such as Will.0.w1sp (Woolford & Guedes, 2005) and Côr (Guedes, Ula li & Woolford, 2003)

More information is available at the NYU Abu Dhabi Arts Centre website at: https://www.nyuad-artscenter.org/en_US/events/2018/jinn-carlos-guedes/

Animation: Kirk Woolford
Music: Carlos Guedes

Choreography & dance (live): Kiori Kawai
Choreography & dance (video): Nella Turkki
Flutes & voice: Cristina Ioan

Cinematography: Saguenail
Lighting design: Simon Fraulo
Costume design: Judi Olson
Marimba, vibraphone, & glockenspiel (recording): João Dias
Sound recording & technical assistance: João Menezes
Stage Management: Tegan McDuffie

Particle dancer excerpt from Me the Machine music video.
The full video is available at youtube.com/watch?v=N0lCL2hpRPM

Video concept: Imogen Heap
Director/set design/co-editor: Ersinhan Ersin, (Marshmallow Laser Feast)
Co Director and Editor: Leo Fawkes
Producer: Marta Sala Font
Production management and design: Liz Berry
Visual/glove integration, sound: Adam Stark
Visual/glove integration, Kelly Snook
Visual Artist, Particle man visuals: Kirk Woolford

Full credits available at youtube.com/watch?v=N0lCL2hpRPM

Moments in Place is a series of site-specific augmented reality performances, created for the Brighton Digital Festival, inviting visitors to consider movement qualities of different locations in the city of Brighton as well as the range of artworks in Brighton's streets.

Each of the performances were recorded on site using portable motion capture systems. When phones or tablets are pointed at select urban artworks, a performance is rendered live in 3D allowing the audience to walk around and explore the relationship between the performance and location.

The original mini-site for this project is located at: http://www.bhaptic.net/moments/

Double-tapping on the screen while the app is running brings up menus displaying "what to look for" and "where to look" in order to help find the appropriate artworks. Alternatively, users can select "Just Dance" to view one of the performances without tracking.

The project invites viewers to think about movement qualities of various locations. Some of the locations are quiet and inspired slow, reflective movement while others are very hectic -- full of shoppers and tourists.

You should try to visit at least a selection of locations. However, if this is not possible, or if your are not able to visit Brighton, you can point your phone at any of the webpages in the "what to look for" section, and the phone will play the performance associted with the location.

Alternatively, you can go through the menu and select "Just Dance" to watch one of the performances wherever and whenever you wish.

Concept/direction: Kirk Woolford
Sound: Carlos Guedes
Movement: Ailed Izurieta, Patrizia Penev, Marjolein Vogels

ABSTRACT
Will.0.W1sp is an interactive installation exploring our ability to recognise human motion without human form. It uses particle systems to create characters or “whisps” with their own drifting, flowing movement, but which also follow digitised human movements. The central point of the environment is a 2x6m curved screen allowing the whisps to be projected at human scale while giving them enough to space to move and avoid visitors through the use of a combination of video tracking and motion sensors. If visitors move quickly in the space, the particle flow becomes erratic. If visitor moves suddenly toward the whisps they explode. The installation system uses custom particle targets and funnels instead of traditional emitters. It performs realtime motion analysis both on the prerecorded motion capture sequences and the movement of the audience to determine how to route the particles across the scene. The motion vectors are simultaneously fed to an audio system using audio grains to create sound flowing in synch with the imagery. Will.0.w1sp invites visitors to chase after virtual, intangible characters which continually scatter and reform just beyond their reach.

INTRODUCTION
Will-o’-the-Whisp, Irrlicht, Candelas, nearly every culture has a name for the mysterious blue white lights seen drifting through marshes and meadows. Whether they are lights of trooping faeries, wandering souls, or glowing swamp gas, they all exhibit the same behavior. They dance ahead of people, but when approached, they vanish and reappear just out of reach. Will.0.w1sp creates new dances of these mysterious lights, but just as with the originals, when a viewer ventures too close, the lights scatter, spin, spiral then reform and continue the dance just beyond the viewer’s reach.
Will.0.W1sp is based on real-time particle systems moving dots like fireflies smoothly around an environent. The particles have their own drifting, flowing movement, but also follow the movements digitised human motions. They shift from one captured sequence to another – performing 30 seconds of one sequence, scattering, then reforming into 1 minute of another sequence by another dancer. In addition to generating the particle systems, the computer watches the positions of viewers around the installation. If an audience member comes too close to the screen, the movement either shifts to another part of the screen or scatters completely.

OVERVIEW
The human visual system is fine-tuned to recognise both movement and other human beings. However, the entire human perceptual process attempts to categorise sensations. Once sensations have been placed in their proper categories, most are ignored and only a select few pass into consciousness. This is why we can walk through a city and be only scarcely aware of the thousands of people we pass, but immediately recognise someone who looks or “walks like” a close friend.

Will.0 plays with human perception by giving visitors something that moves like a human being, but denies categorisation. It grabs and holds visitors attention at a very deep level. In order to trigger the parts of the visual system tuned to human movement, the movement driving the particles is captured from live dancers using motion capture techniques. While the installation is running, the system decides whether to smoothly flow from one motion sequence into another, make an abrupt change in movement, or to switch to pedestrian motions such as sitting, walking off screen, etc. These decisions are based on position and movement of observers in the space.

The choreography is arranged into series of movement sequences which flow through specific poses. It is a mix of shifting approaches and retreats as the system plays through its sequences and responds to the presence and movements of the audience.

SOUND
While the installation attempts to present human movement without human beings, it also pulls them into a sonic atmosphere somewhere between installation space and some space outdoor at night. The sound has underlying melodies punctuated by crickets, goat bells, and scruffing sounds from heavy creatures moving in the dark. All this is generated and mixed live by software watching the flow and positions of the particles in the space.

The sound software is a Max/MSP patch with custom plug-ins written by Carlos Guedes. Because it uses so much cpu-power to generate the sound, it is run on a separate computer. Particle motion data, overall tempo, x, y, and z offsets are all transmitted from the computer controlling the particles to the sound computer using Open Sound Control streams.

MOTION DATA
The installation uses a small a database of motion sequences as the base animation data for the particle dancers. Initial tests sequences were created with a Hypervision Reactor 3D motion capture system. However, the system used for these tests required a great deal of “data cleaning”, or manually correcting of motion data. Sometimes it was necessary to adjust 32 points for every frame of capture data. We decided this was just as time-consuming as manually rotoscpoping data, so we created an intelligent rotoscoping package called “Rotocope” shown in “Figure 1”. Rotocope allowed us to manually position markers over key frames in video-taped movement sequences. Rotocope

PRESENTATION
The project is presented on a custom designed screen. The screen is slightly larger taller than a human and curves slightly from center to the edges. This keeps the images at a human scale while completely filling the audience’s field of vision. At the same time, it allows movement at the edges to shift and disperse more quickly than movement in the center. The screen is much wider than standard high-def format (2.2m x 6.0m) to create more space for the image to shift and flow. It is driven by 2 synched video projectors.

INTERACTION
A close reading of the algorithms outlined in sections 4 and 5 will reveal that the entire character of the Will.0.w1sp system revolves around 2 simple values: the maximum velocity and maximum acceleration of the particles. As mentioned in section 3,  increasing the max velocity of the particles allows them to track their motion targets more closely and the particles begin to take on the form of the original dancer. Increasing the acceleration, narrows the width of this dancer. At the same time, decreasing the velocity while increasing the acceleration causes the particles shoot off in nearly straight lines. In effect, it explodes the virtual dancer.

The Will.0.W1sp interaction system is a separate program which tracks the positions and actions of visitors, and decides how to modulate these two core variables. It also controls where on the 6m curved screen the dancer positions itself.

Inverse Interaction
When many visitors walk into an “interactive” environment, the first thing they do is to walk up to the screen and start waving their arms madly to see the piece “interact”. Will.0.W1sp responds the same as most humans or animals. It scatters and waits for the person to calm down and stay in one place for a minute. It uses inverse interaction. The installation uses interactive techniques not to drive itself, but to calm down the viewer. When the viewers realize the system is responding to  them, they eventually extend the installation the same respect they would extend a live performer.

Tracking techniques
The first version of Will.0.w1sp used an overhead camera and image analysis software to track visitors. This worked as long as the lighting conditions could be controlled. However, it was unstable. When Will.0.w1sp was invited to represent Funcaion Telecom at ARCO’06 in Madrid. A robust tracking system had to be developed which could handle up to 50 people in front of the screen at once, and a continual flow of more than 2,000 people a day.
We replaced the overhead camera with an array of passive Infra-Red motion detectors connected to an Arduino[2] open source microcontroller system We developed a control system for this sensor array using the “Processing” [3] open source programming environment together with the processing “oscp5” Open Sound Control library developed by Andreas Schlegel. The tracking system records motion in 9 different zones and calculates max velocity, acceleration, and character position based on positions and overall motion of viewers. It uses its own timers to suddenly ramp up values and slowly return them to normal when it no longer sees any motion in any of its sensory zones.

CONCLUSION
The installation is very successful in its ability to generate dynamic images which walk the line between recognition as human and other. Viewers to the installation often feel they are in the space with another “live” entity which is not quite human. After initial playing around to see the particles scatter, they will often sit for an hour or more to watch them perform.

ACKNOWLEDGMENTS
Will.0.w1sp was funded by a grant from the Amsterdams Fonds voor de Kunst and supported by the Lancaster Institute for the Contemporary Arts, Lancaster University.

REFERENCES
[1] C. Guedes, “The m-objects: A small library for musical generation and musical tempo control from dance movement real time.” Proceedings of the International Computer Conference, International Computer Music Association, 2005, 794-797
[2] C. Guedes. “Extracting musically-relevant rhythmic information from dance movement by applying pitch-tracking techniques video signal.” Proceedings of the Sound and Music Computing Conference SMC06, Marseille, France, 2006, pp. 25-33
[3] http://www.arduino.cc/
[4] http://www.processing.org/

A real-time multi-player online game based on the world of the mafia. Players begin as lowly thugs, doing drug runs and climb the criminal career ladder by accumulating wealth and status to become Godfather of their own gang.

Client: Woedend

Role: Redesigned interface and MySQL database schemes, added new security, puzzles, and mini games

The game has been relaunched with new web and mobile interfaces at: https://www.mobstargame.com/

Eccky was created in August 2005 by Dutch developer Media Republic in association with MSN in the Netherlands. Eccky has characteristics of life simulation and virtual pet games. The gameplay of the first version of Eccky involved a virtual baby, or Eccky, which was born on the basis of information derived from both Eccky user players. Eccky used an AIML chatbot and MSN Messenger for chat between users and the Eccky baby. In 2006, Eccky became an independent company as a subsidiary of Media Republic.

Client: Woedend for Media Republic/MSN

Role: Sole programmer developing a mini-game for the Eccky network.

Translation: Eckky is in love!
Except that Eccky doesn't dare to tell the girl or boy that s/he has a crush on them.
You need to pass notes to Eccky's new friend telling him/her that Eccky likes them. But be careful, other the other kids in the class are trying to pester Eccky's heart-throb with their own teasing notes.

The game mechanic is note-throwing. The goal is to throw notes that land on Eccky's friend's desk, while also knocking knocking away the notes from the students.

The game is written in Flash, with live connections to the Microsoft Network (MSN) servers for Eccky's gender, appearance, and experience.

Delivering the equestrian life, from sports and leisure riding to horse care, My Horse and Me is a unique gaming experience for anyone with an interest or passion for horses. The game features the most accurate horse models and animations yet realized in a video game, alongside a rewarding game play experience, all set against a backdrop of indoor and outdoor environments. My Horse and Me has a variety of game play modes and options to give players an authentic experience. The Championship mode lets the player take part in competitions at indoor and outdoor locations around the globe ranging from rustic stables and classical riding schools to world-class tournament locations. A series of mini-games offer a variety of game play experiences alongside rewarding horse care game play and extensive customization. This title also offers both first and third person camera modes, putting riders right in the saddle and creating a perfect training tool for practicing disciplines that riders face in real life.

I conducted the market research into "Girl Games", and co-directed the original design team to create one of the first games released on the Nintendo Wii

Woedend Team:

Game Designers: Jochem van der Spek
Marc Schmidt

Project Leaders: Kirk Woolford
Bas van Berkestijn

Art Direction: Paul Coops
Jochem van der Spek

3-D Artists: Jurriaan Hos
Mischa Rootsaert
Mattijs van der Valk
Bart Janssen
Paul Coops
Ben Vergeer

Character Design: Hans Pieko

Dynamica: Jochem van der Spek

Technical Research: Kirk Woolford
Jochem van der Spek

Sound: Soundware Amsterdam

Traffic: Rianne Zandstra