Overview
The Work "Navigating and Negotiating Sound Architectures of the Night" is
a major evocative multi-media black box theatre work. Its hybrid form includes 3d
architectonic models that are both virtual and physical; digital video; a generative
visual system including 4 real-time projections; 10 live musical improvisers; a
generative soundtrack; a recorded spoken poetic text; and interactive interface
table enabling multiple interactants to explore the work simultaneously.
Description
The work engages with the poetics of night through an interactive structure of 84
abstract architectural models that are explored over time through multiple sonic /
architectonic movements; through placement and spatial movement /
displacement; as well through architectural abstraction. A specially designed
glass interface table enables participants from the general public to interact with
the work and become part of the performance via this intuitive multi-user
interface. A computational sensing system and specially authored computer code
enables one to juxtapose projection-like texture maps of night-related
architectural / city-scape imagery, forming large format virtual / architectural
landscapes presented as 4 large scale projections in the space. Each projection
contains the view visible from a particular side of the table. A seven movement
generative/interactive musical score is also driven via the interactive choices of
the participants; and a series of 10 live audio improvisers also add additional
layers to the interactive musical score. Multiple architectonic sculptures become
part of the "set". A poetic text related to the night theme is also enfolded as part
of the environment. The work is highly interdisciplinary and students from over 5
departments at Rensselaer Polytechnic Institute have been participating over the
course of the year.
Pieces – Physical Interface Units
Each of the 84 physical interface architectural primitives (physical architectural
model shapes) are robust in nature — many people interacted with them during
the performance; each piece is recognized by the system through a unique
symbol on the bottom of the architectonic form. An additional set of objects can
be placed next to the models and create architectural abstractions in real time on
the display screens. This makes the work extremely flexible in terms of real-time
exploration of architectonic forms and related texture maps. The architectural
models have been made of many different materials – wood, metal, plastic (3D
printed) etc. Usability tests were done related to which materials would work
Best.
3d Projection – A set of computer programmers worked in conjunction with
“Model” designers.
For each physical piece an isomorphic form was made with 3D modelling
software. The programmer facilitated “parallel” spatial moves in the “projected
space” to that of the movement of the object on the interface table. A “Night”
video landscape was also projected behind the 3d Models/texture maps. Thus a
kind of 1-1 relation was formed and brought to life via computer code. For each
piece a set of texture maps was developed.
A series of music loops were developed by Seaman before the performance,
working in tandem with student members contributing sonic materials that were
folded into the mix. These loops were “attached” by the programmer to each
object. Each time an architectural model was put down on the table that loop was
played back in the room. The room reproduced the spatial positioning of the
objects on the table and as participants moved the objects, their loop moved in
physical space forming a dynamic real time spatial mix. Each of the 7
movements had 12 differing loop/objects and functioned as a generative
soundtrack. Docents delivered the new objects and cleared the table for each
new movement.
When the participant/interactant placed the object on the glass interface surface,
the image appeared in real time on one of the screens from a particular
“perspective” that the programmer had authored. Texture maps also appear on
the surface of the object at that time as presented on the screen. As discussed
above the sound had a positionality in real space. When the participant moved
the piece the sound actually moved to a corresponding location in physical
space. A dynamic relationality of the pieces was thus achieved though both sonic
and visual spatial means. The goal was to make a working system where many
pieces could be put on the table and interacted with via multiple interactants
simultaneously enabling quick dynamic architectural design capabilities — thus
the work functioned in part as a real-time multi-user “sketching” tool as well as
artwork generator.
Once this goal was achieved, we authored higher level functionalities that related
to the proximity of one object to another, and/or the functionality of one set of
objects that could functionally abstract the architectonic qualities of the original
object – e.g. make differing algorithmic abstractions of the objects including the
following— making the selected objects bend; multiply; deform or twist; as well
as make them more or less transparent; change the scale of the objects that are
projected; and/or change the height of the objects.
The Interface Table
The interface table had a sensing system that read the symbols affixed to each
object via an infrared camera. The table was robust and inspired in part by the
initial open source specifications provided by Todd Berreth (Seaman’s
collaborator at Duke). The table was later fully re-designed by the team. The
room had 4 major screens as one component of the work each providing a
differing view of the architectural models covered in texture maps, with an
additional slow moving/pulsing video texture map as background. This physical
interface table empowered dialogue between participants standing adjacent to
the table who in real time manipulated architectural models/projections/sound
positioning. Over the course of the evening literally hundreds of participants
explored the system!
Being the main focus, the table was placed in the center of the room. The 12’x16’
screens were placed 8’ above the ground, tilted at an angle that was comfortable
for audience members to see. The luminous screens together formed a large
central space that with the table created a space that was the focus of the
performance, though the participants were allowed to move freely within the
room. The projectors were placed in the corners of the room, rear projecting onto
the screens.
Live performers in the Space / Generative Audio system
Pauline Oliveros worked with a group of improvisers who were distributed around
the edge of the room. This group provided some audio materials to Seaman for
the construction of the loops. These audio materials consisted of improvisations
recorded alone and/or in groups; as well as field recordings at night – drones (the
sound of lights or transformers; cars over the surface of a bridge creating pitches,
instruments and sung aspects recorded from a distance; Seaman made a
generative loop system with additional instruments that was designed to work
together in any generative combination of the architectonic movements. These
loops were attached to the architectonic “model pieces” described above. As the
pieces were manipulated on the interface tables individual performers improvised
with these loops distributed across the space. Thus both the loops and the
performers had a spatial position in the room. Performers positioned themselves
in relation to each other – e.g. working closely together to make a certain kinds of
sound --- and also functioning at a distributed distance to each other.
To give the performers a defined space, we created canopy structures that hung
at various heights over the performers. The musicians needed to be able to see
each other in order to give cues while improvising, and most also needed power
sources. They also couldn’t be too close, crowding the central table. We spaced
the performers out around the perimeter to fulfill these requirements. Small
architectonic enclosures were designed. These canopies were dimly lit to have
an ambient glow. The table also had a light over it that would come on in
between each set to signal a new movement.
The Changing of the Pieces as a performative element
I made 12 sonic submixes of these tuned loops for each of the 7 movements –
one for each object. Thus one series of these loops/objects were brought out and
placed on the table. These were then completely removed and a new set were
brought out. Thus new sonic keys and subtle rhythms could be provided over the
course of the seven movements lasting approximately one hour. In this way we
could achieve movements in the work – one for each night of the week.
The goal was to get one set of objects/sounds functioning that produce a selforganizing
system with variation between the audio work produced via the sonic
spatial mixing of performers and loops. Once this had been facilitated 6
additional “sets” were devised and a series of ambient plateau-like sonic states
functioning at different levels of intensity were authored.
Video Materials
Video images of night both still and moving were recorded by both myself and
the students. These were reduced to HD for playback (and saved at even lower
resolutions to function in conjuction with the programmer’s needs in relation to
the specifications of the numerous texture maps.
Poetic Text
Seaman wrote and recorded a poem which was presented in the space in two
locations via highly focused speakers from above. Thus, one “found” the poetic
text while walking around the room. Only when directly under the speaker did the
poem become part of the mix.
Programming
The programming was built using the Unity engine. Use of both open source and
new code was explored. A central machine would read the IR video image via a
camera which was housed in the bottom of the interface table pointing upward;
and the system processed the symbol token for identity, location and orientation.
Identity refers to a database that contains, in this case, 3D models, texture maps,
and sound. Most tokens refered to 3D models but some were modifiers that
transformed objects that they were adjacent to. The database for modifiers was a
different one.
A central machine communicated the locations of the objects and orientations to
four other machines, each running a copy of the unity environment. Objects were
then placed and rendered in each separate system, this enabled the unique
camera locations to be designated in each case with the coordinated yet unique
perspectives shown on the large displays above the tables. This process also
resulted in a system that was robust to glitches/failures during the performance.
Sound files all run continuously in the background in an initial silent mode.
Placement and removal of objects on the tables turned up or down volume with
fast controlled attack and decay envelopes. This allowed the sound files to
remain coordinated. I have enclosed the coordinated movements 1-7 —this
generative coordination is achieved by playing back all of the 12 loops for a
particular movement in sinc. The object’s placement on the table also
determines the apparent location of the sound within the black-box theatre using
a novel sound rendering technique by Samuel Chabot/Jonas Braasch detailed in
the MS Thesis: SPATIALIZED SOUND REPRODUCTION FOR TELEMATIC
MUSIC PERFORMANCES IN AN IMMERSIVE VIRTUAL ENVIRONMENT.
Documentation
Both still and video documentation of each of the architectonic movements were
recorded from multiple positions in the space at EMPAC where the work was
performed. I have enclosed the first 3 minutes of movement 1 as documentation.
Each movement was similar in structure where the models were delivered, and
put on the table one at a time. The actual duration of the 7 movements is variable
but at the moment we have a 54 minute version. The improvisers that played
along with my generative audio files were students studying under Pauline
Oliveros. The performance was well attended. Sadly, Pauline Oliveros died
shortly after this project was completed.
Navigating Sound Architectures of the Night
Swimming in the night
In the sound
Across the lake I hear a loon
A light shimmers a line of abstraction in reflection
One can hear the shape of the landscape
As sounds echo across the night from shore to shore
Cove to cove
A sound architecture, a landscape reflection
The waves are refracted in the wind of thought
Transparent light walls speak in muted voices
I am a child, blindfolding myself in darkness
Approaching blindness with intention
The thought of blindness
To understand
To learn my reach, my grasp of space
My bodily limits
I learn the room, felt meanings on memories
A history of relations
bumping over the glass and the water spills
So different
The sound of the heater releasing steam
I grasp the architecture as I fall in relation to it
In a Paris hotel room, I forget the steps having not learned the space
And returning back in the blackness
I fall
I feel the pain of my leg bending back, broken, laying now on the floor
In the deep black depth of the night
Waiting patiently for the light
The window forms in the distance
A figure moves across a room
Color perception diminished
A perfectly lit room framed by the dusk
The clarity of the air, a disturbance,
transmits the light in a crystalline manner
clearly the dead of winter, extreme cold
Light illuminates the dance
Steam forms shape shadows
as two workman cast a perception of their frames
Unknown noises abstracted by distance and wind
Move in and out of the audible range
In resonance with the shape of doubt,
my sense projections filtered
in the light of the evening’s noise
at first it appears to be a figure
I project onto the landscape
a history of observations
Into the darkness thought moves
proximity, a relationality, a false projection
An illusion of the night
A quiet disillusionment unfolds
A poem of light and space
Speaking of the night
The light states are captured in this instance,
Housed, transformed, and released
With sound architecture framing these dark transparent meshes
projected sketches, made of mutable light motions
These quietly moving forms, form a space of negotiation, of thought passages
As they play out across the night’s figures and architectural domains
Where will we end up after these navigations?
A new place in thought.
The chess-like moves have had the rules shifted,
or removed --- they are open
So negotiations of light reflect the navigations
the night sounds are triggered and released, each unfolding a spatial relation
setting in motion living responses
reactions flowing
patterns of resonance transgress the dissonances of darkness
Feeling the way around the darkened room
picturing your face
From memories of touch
Eyes closed
One remembers this touching moment
The limits of a body of thought – a body of memories
The light arc of experience
I have found my way here navigating negotiations
Memories of the movements within
The dark spaces of my nature
These multiple lives
in thought and action
In a vast architecture of darkness,
Navigating meshed virtualities
the losses, with sound enterings,
the back and forth, turning around,
turning in on itself
The quiet internal conversation.
The night is a veil masking true detail
Again, mistaken in our projections
overlays
doubts made palpable
The architectonic forms of thought
Are here and gone
On the tip of the tongue
behaviors are rendered, remembered.
The skew embrace
The body’s positionings in space
The movements and proximities are navigated
In the dark room
Focused rays render the moment
Light abstractions, memories and realizations
Are rendered palpable
After years of being locked away
The space of an association is released and meanders
Open to this negotiation
This speaking across boundaries
Triggers
as the world unfolds in the moment
Out of these spaces of dark silence.
These sound architectures
The architecture of thought and remembrance.