Women will discover the unknown. Will her word be different from ours? She will discover things that will be strange and unfathomable, repulsive and delicate. We shall take them from her and we shall understand them. (Nancy Mauro-Flude)
In this project, Interaktionslabor 2006 partners with teams from other locations in Europe: independent media performance groups and media arts centers. The concept and the dramaturgy for the research production, to culminate in a public/distributed live performance in Athens (14 September 2006), is described on the pages of the "i-MAP" website. It is worth pointing out that such a European partnership project, linking artists and media arts centers in the east, south, southwest and west (if we re-introduce geo-political or cultural-geographical markers), generates its own process as it undergoes organisational, financial, logistical and artistic formation, over a series of meetings, constant communications, maillists, websites, and locally arranged workshops that bring the participating members of the project together.
The institigators of the transeuropean "I-Map" project, Amorphy, are based in Athens, and thus it is logical that the cycles of workshops will return, at the end, to the city where the ideas for the project were born. The live installation of the new work is currently titled "See you in Walhalla." As always in media and live art processes which involve a large group of professional artists from the performing arts, visual arts, computer science, design and digital media, there are languages and areas of expertise to cross and artistic perspectives to combine. Process is always a process of negotiation and of course collaboration. Research into the programming of interactive game environments as live performance (with webcams, with sensors/wearable computing) is research into performance but also research into the (design) relationships between media and genres, between aesthetic positions and understandings of the particular function or language of a medium. At this moment, I am not even addressing any larger philosophical and ideological issues, and I have not yet commented on the nature of the cultural collisions in such a trans-European project, even if one could also get the impression, watching behaviors and conversations, that we share a common advanced postindustrial culture of the west (and its naturalized communications systems and systems of mediation). But not everyone may share this western outlook or professional code, and we have already had visitors from local regions who are unfamiliar with the code, and we've had visitors from different cultural contexts (Iran).
But even if technical processes (the complex development of the programming of the patches for this interactive performance, the long and detailed process of adapting the sensors to the requirements of the work/performance, the inclusion of webcams or live telematics into the interaction design) and artistic rehearsals, dramaturgy developments, scripts, the film shoots, editing, composing of sound tracks or real time patches for live mix, the physical vocabularies, and the whole range of visual scenographic elements (screen projections space, "stage," ligthing, costumes, etc) are correlated and integrated, the critical reflection is still necessary, and reflection on integration, on changes, and on emerging aspects of the contents needs to be re-negotiated all the time in an open and peer-to-peer relationship between collaborators. Thus, at this moment of writing, the title "See you in Walhalla", the content direction, the semantics and the 'characters' in the game are all under scrutiny. In these last days of July, the second week of the production workshop at Interaktionslabor, the assemblage of all elements into a digital/interactive live performance provides the first opportunity to see the whole emerging work. In this sense, it is quite correct to see the Interaktionslabor Göttelborn, as the southwest german "iMap" partner, in the role of the "testbed" --- here the simulation of all elemets of a distribited work can be accomplished, all Real Players, Avatars, Confused Players, Screen Avatars of the "Walhalla" performance are meeting now: in front of the screen in the 3D/4D/5D space, inside the screen, on webcams, off webcams, inside our building and outside:
Performance space: Interaktionslabor Göttelborn, webcam players outside the Gray Gallery and triptych with Avatar perfomer inside the gallery
Sensordance, interactive game, webcam dramaturgy.
Extending previous studies of physical camera, sensor choreography, and interactive design for real time networked performance, the lab this year uses a combination of indoor and outdoor spaces at its Performance Gallery. The commentary I propose to develop here concerns the experimentations with avatars, "sensor dance", "sensor choreography," and real time interaction design for performance with digital 3D worlds combined with web streams.We invite feedback. (e.g. testbed 4: http://interaktionslabor.de/lab06/gase.htm), and hope to extend this discussion into the internet communities.
The term "sensor choreography" may not be the right term. But I'll be glad to define more closely, in debate, what I see happening in our workshop --- as we move from the general daily rhythm of warm ups, body mind centering/yoga based exercises, and somatic techniques to movement-choreography rehearsals with three performers (dancers/actors), to the performance with sensors. The three performers each rehearsed individually with a dramaturg, which is perhaps unusual as a method, but I think the connection in this production, and in this research, builds on the following areas, and dramaturgical decisions are necessary in each of them:
1. Theatre / performance -- devising with digital media (a dramaturgical script has been written by the team directors: this script is a complex piece of sketching actions on many levels (screen projection of the triptych/video actions, webcam actions in the remote sites, 4 characters [Player 1, Player 2, Real Avatar, Screen Avatar] , direction of local and distributed action; sound direction, text/voice-over direction, scenographic direction, interactivity direction, precise lighting design).
2. There is scenography, as there is a local venue where the distributed performance will be first performed (in Athens). Stage design, screen design, and lighting design are precisely configured.
3. Interaction design:
----level A: digital video or 3D animation on three separated screens creating immersive world (the screen architecture is 12m x 3 meter; the playing area for Real Avatar / live dancer is 2 meter wide and 12 long). Hundreds if not thousands of film scenes have been shot on location, in a special triple camera set up, many tracking shots, now edited in small clusters. Amount of digital film material roughly 1 terabyte.
The interaction patch is written by programmers on open source software (PD)
----level B: sound design/music (not interactive, it is created by composer and performed in real time synthesis through a combination of patches)
3. Sensor design
---- level A: physical sensor design for various types of sensors (orient, flex, tilt, and rotation sensors; switch/clicker; turn/resistor-switch sensor; heat sensor, photosensor) --- which are all from the same family of accelerometers and sensors, looking slightly different....... providing different information depending on where they are attached to the body or rub on the flesh in the garment or are moved and pressed as you would when you touch garment fabric or skin. The sensors we use in rehearsal come from I-CubeX (Wi-miniSystem) and La Kitchen (Kroonde Gamma, a wireless, high-speed, high-precision data-acquisition system dedicated to real-time applications/performance using embedded sensors).
The transmitter sends the data from the motion to the computer laptops, via wireless Bluetooth transmitter (I-Cube X) or OSC/UDP (Kroonde), and on the computing side the information/data is received into PD patches (we are also using Max/Msp patches).
--- level B: programming of the interfaces (motion data/sensorial data control of what we could call, citing Manovich, the "image instrument", i.e. the digital images and data and thus the game world).
4. Sensor Choreography; Movement Behaviors
To come back to my initial point, the movement vocabulary, involving our Real Avatar (dancer) and the remote Players (streamed into our triptychon video projection scenography) was rehearsed and developed for the characters in the game world (which is, largely movement and behaviors), while in the first week our engineers and programmers were still working on the interaction design. First 6, then 14 sensors were devised (an additional set of click switches was custom built and sewn into hand made "gloves") to be worn by dancer Ermira Goro. A "mapping" of the sensors and their anatomical positions and functions on the body was drawn.
Since Sunday, July 23, Ermira has been dancing the scenes of the game world choreography ("Walhalla") wearing 14 sensors. That is to say, she is re-choreographing her movement choreography incorporating the sensors on her body which allow her an immediate, direct relationship to the virtual worlds (digital film and animation) in front of which, into which, she literally moves and navigates.
As with an avatar one would navigate in a game through console or keyboard, the dancer in our production is navigating her character into the digital world, and she is also responding to the programmable interactions she does not control, such as the appearances of the other players streamed in from the webcams in Amsterdam and Sofia.5. Live web stream dramaturgy
This element of the live performance involves the incorporation of remote performers in either controlled (indoors) or uncontrolled (outdoor urban) locations which are monitored or surveilled by webcams whose live streams are combined into the digital projective media seen by the audience and, potentially, seen, experienced and manipulated by the performer at the receiving end (Ermira). The creation of the live webstreams can be two-fold: (a) the remote performer can be distant from the camera and perform indoors or outdoors knowing that any action (intended or unintended/contingent) can be captured and transmitted; (b) the performer can operate in close proximity to the webcam and manipulate the camera herself or use camera manipulation by crew members. In extension, the performer, as it was tested by Nancy Mauro-Flude in our lab, can hold the (wireless or wired) webcam in her hands and create a very specific movement-stream of images for transmission over the internet.
Interaktionslabor Göttelnborn functions as a "testbed": during the following days we are putting together the various elements of the digital performance, and simulate the remote Players by having them perform here with us, but outside, in the Coal Mine, in public squares we have lit and to which our Wifi network connection for webcasting reaches. Later, the two webcam performers will depart to Amsterdam and Sofia and work with us remotely. Interestingly, the telematic component of "organization" now enters the fray. Since the dramaturgy obviously has a sequence of scenes and cues, these cues have to be transmitted via chat lines and chat operators to the remote Players/Performers, they need to hear the cues, and enact their actions in precise range of the webcams following a specific timing. So for the remote sites, each performance requires a network director, a chat operator, and a stagemanager or mediator. If the remote Player is out in public squares, the cues will be signaled to them via mobile phones or acoustically.
More elements of the design, choreography, film, webcam operation, editing, sound, etc can be brought into the discussion, but in the next section, I will add some comments on the notion of the avatar as game character in this performance.
The Game World as System
The streaming, telekinetic performance (“See you in Walhalla”) is written and created following the logic of a computer game. The audience will not have a direct interactive experience of the game; the Player and the Avatar will stand in.
Downloaded and composited by the Player, the avatar is a creature of the game design world or, in extension, of the game industry. One can even argue that the Avatar is an "industrial worker". The most significant difference to a computer game played via console or keyboard is the fact that the interaction in this live work is externalized into a 3-D or 4-D environment - the avatar leaves the flat flat screen animated computer-generated world. The work generates the new genre of a live interactive a-game (artificial life).* The avatar is embodied by a dancer performing in front of the triptychon film projections and animations. In the 18 scenes devised for the a-game "See you in Walhalla", she encounters real people and prerecorded life on streets and occurrences in parallel/combined/mixed urban realities. She hears the voices of people in Sofia, and the street noise in Amsterdam. The webcams create a live dimension that is almost completely paradoxical in relation to the notion of a "programmed" game that allows only certain interactions between Player and game world. But here the game world is at the same time a digitally re-created realist world of cities intermixed, flowing together. All digital film material was filmed on location in Athens, Amsetrdam, Sofia anmd Göttelborn.
Through collaborative inter-media creation process, developed simultaneously by teams of artists in three different locations, a dramaturgical structure for the a-game is created. In terms of the thematic and functional content, we can also speak of a "sensordance game", a live 3D computer game, an avatar dance, and generally of an interactive art work. On another level, the work represents research into artificial life, and thus addresses interdisciplinary science focused on artificial systems that mimic the properties of living systems. In the larger context of such research on a-life art and emergence, the project reflects on generative processes which connect artificial systems (computer animations and synthesis) and living matter. The artificial system accepts biofeedback and is responsive: the software environment can be inflected by the human avatar.(Sher Doruff speaks of biogrammatic experience in this context).
This telekinetic performance event will be simultaneously presented in three European venues (Athens, Amsterdam and Sofia) on September 14, the "main" stage being Athens. These venues will have fully interactive capabilities, allowing for live control of media in all three locations, creating a shared virtual environment, guided and utilized by the participating artists. The teams have come to Göttelborn for a “testbed” phase, rehearsing the new work and completing the collaborative programming environment. One of the questions raised in the feedback sessions concerned the role of the avatar, and the "character" of the Player (the average guy who loves gaming and plays this particular game). Even more difficult to understand, at first sight, is the character of the Confused Player who perhaps stands in for a confused audience which is first invited to adapt a game scenario as their reception of the work. They see how the Player downloads the Walhalla game and starts "designing"/building the avatar, here performed live by Ermira Goro. In other words, the audience is invited to enter into the game system as a system of organizing perception and of steering the participation. Each game, as we know, has rules and regulations, it follows a protocol.
But the game character of synthetic worlds is subverted or reversed -- the avatar is not walking through CG words., she walks through streets of AthensAmsterdamSofia, European cities merging, flowing together, public squares, alleys, canals, trams, walkways, buildings, people walking across public spaces, private spaces, nights and days, days and nights. The protocols of game worlds are replaced by the protocols of urban life.
* cf. Mitchell Whitelaw, Metacreation: Art and Artificial Life. Cambridge: MIT Press, 2004.
The System as Game
How is the performance reorganizing perception/proprioception? On one level, the performance of the avatar is deceptive, as the movement vocabulary used by Ermira Goro is closely representational of actual computer game avatar design-behavior. Ermira performs like an animated figure of a game. During the prolog, she is initially "stuck" in the program as a pure videopuppet or "animation object" waiting to be activated. She is stuck in a holding pattern. Not "lifeless", we notice very small twitches in her body, a nervous jolt of her shoulder or her hand, almost perfectly gamelike in behavior.
When the Player downloads the game and activates her, she slowly comes to life. She begins to face the virtual environment into which she is sent and which will stream towards her and past her. She appears controlled by the game engine, and her path seems set. She enters into her first movement behavior -- walking.
As we hear a voice intone a commercial text about "healthy walking," the avatar performs a walk that slowly animates her whole body from head to toe, and it is the coordinated and subtle motions of her head, shoulders, arms, hands, fingers, pelvis, legs and toes that initiate the particular compositional rhythm of the game system "building" itself up, so to speak. Her motions "release" segments of the city environment, and as all of these segmentations of her body are sensorialized (14 sensors attached to her body transmitting data to the computer program devised by the project team) and thus intimately associated to what the avatar will recognize as "connected" to her motions. This scene is called "isolation downloads" in the script, and the dramaturgy for the opening scenes thus provides a logical and systematic convergence of choreography and digital imagery. The avatar slowly and continuously seems to be composited into the building blocks of the city environment, stripes of images appear and disappear, images at different height levels appear to respond to particular motions of her body at that height level. When the whole picture is complete, the avatar is walking in a graffiti environment street before exiting into the "real world." The visual system thus begins to introduce, from the beginning, a segmentation and slicing of the planes as well as a movement of images that can have different tempi -- the triptych of screens thus comes alive as a highly unstable, flexible, moveable and nervous system.Our initial connection to the avatar is purely through the kinetic rhythm of her forward motion, and the visual flow of the fleeting urban images that seem to "jump", at first, jump at us like slices of life, then move towards her (and us). She appears immersed in the "journey" through the game world. Gradually, however, we begin to notice small differences. First walking forward only, her motion now also includes a "swivvel" - she turns around her axis and also to her left and right side. The most striking aspect of the movement Ermira has created for her character is the walking on the same spot, giving the appearance of walking forward into the images but in fact remaining in place (like on a treadmill); often she appears to be "floating" and the kinetic movement of her whole body is carefully articulated. It is interesting that in the opening sequences of the performance the game figure (avatar) thus also evokes a parallel system (sports) that will be discussed later in this commentary. I call this opening sequence her "training session." Another dimension of the "system" is the visual environment formed by the digital film images, which from the beginning evoke the scenario of city streets, alleys, sidewalks with houses lining each side of the central avenues. The filmed material is composed with an extraordinary tuning and conflation of parallel tracking shots (filmed on location with three cameras mounted on a vehicle). The smooth gliding images always evoke a sense of flowing, drifting through the urban landscape.
Movement, therefore, is the primary dimension of the game in these opening scenes. The avatar is learning how to move through and navigate her environment, she is "sent" on her way.
In the following scenes, her further "learning" and orientation behaviors start. Our avatar begins to engage her environment, and become more responsive to it. In systemic terms, this training sequence also illuminates processes of codification (transmission in the sugn systems of symbolic representation). All processes here, imcluding the avatar-dancer's physical movements, must be considered medial transformations, and it will be of particular interest to observe how codification of gender difference, of emotional content, of physical mobilization, is achieved.
excursus: sports and late capitalism
click to continue
