Final Tests

Today I access the NUA machine for the last time before taking them across to the exhibition venue. I have a number of critical tests to perform and then I shall finalise the exhibition content itself.

With me today I have

  •  3 x HD webcams
  • 3 x USB extensions (10m, 15m and 20m)
  • 3 x simple USB MIDI interfaces
  • 1 x 4 bus USB MIDI interface
  • 1 x laptop running Ableton Live

Previously I had run into an issue with one of the key Quartz Composer (‘QC’) plug-ins not working which was probably due to the machines having being upgraded to OS X 10.8 from 10.7 – many of the QC plug-ins I intend to use are not supported on 10.7. At my request, the college has reverted the 3 exhibition machines back to 10.7 but there has been some confusion over the set-up I require so I am not 100% certain yet that the machines have the necessary software to run the exhibition content.

Hopefully I will then find that the Carousela plug-in works as it does on my dev machine. This plug-in is crucial to the ‘Neon Dance’ piece as it enables outline tracing as a series of geometric points which can be processed to create the Neon Dance outline effect. I have been working on the basis that the plug-in may not work, in which case I have an alternative approach in place (‘Neon Dots’) although unfortunately this is an inferior  approximation of the original.  So the result of the Carousela plug-in test will determine whether I can use Neon Dance in the exhibition or instead make do with Neon Dots.

Next up are the standard calibration tests I have developed to assess machine/webcam performance. I am particularly concerned about the performance of the webcams via the USB extension leads – I recently performed a software update on my dev machine which resulted in a significant loss of USB speed, to the point of the webcam plugged in via the USB extension lead being unusable. From reading around on the web, it seems that the operating system version of the host machine has a major impact upon USB performance. If the USB extensions prove to be problematic on the updated exhibition machines, I will face a serious configuration problem at the exhibition gallery which I won’t be able to solve without discussion with the venue manager, Richard Fair. I did discuss this prospect with him a couple of days ago via email and it seems that there is no easy alternative set-up, so I sincerely hope this will not be necessary.

Assuming all goes well so far, I will then run through each piece individually on the 3 machines. Next comes the final testing of the 3 exhibition master QC files for which each exhibition machine will require its own MIDI interface. I have created a version of the master suitable for each of the 3 machines (to be known as A, B & C). Each exhibition machine is instructed to play a particular piece or ‘vignette’ by the audio machine running Ableton Live. Instructions are relayed via MIDI. In return, while each machine plays its particular variant of the current vignette, it relays real time information (based on user interaction picked up via webcam video input and audio input in the case of the Rutt Ettra piece) back to the audio machine, also conveyed as MIDI  information.

The final outcome will hopefully be to prove that everything works as expected and to validate the master exhibition pieces. I know that much of the calibration to optimise each piece can only take place in the actual venue as it is dependent upon factors such as screen behaviour and lighting. Therefore it doesn’t matter too much what the pieces look or sound like at this stage – just that the mechanics of each are in good working order.


  •  #1 – ensure machines have correct software
  • #2 – check whether the previously misbehaving Carousela plug-in now works as expected
  • #3 – run calibration te including testing webcams plugged in via USB extensions
  • #4 – test individual pieces on each machine
  • #5 – test multiple machine set up (including all MIDI devices)


After time consuming issues with some USB extension leads not working in conjunction with some computers, all tests have now been passed. The next steps are to get the equipment into situ and set-up.

Vignettes are as follows (not final names!) with descriptions for good measure –

Neon Dance

Any significant  movement in front of the screen results in an outline being traced – typically this will match the viewer’s body eg arms being waved around. The tracing is approximate and glitchy, sometimes resulting in an unstable outline which in itself is quite interesting to watch as it momentarily flares out to incorporate bright features such as lights and windows. The lines are traced in a single ‘neon’ type colour which is run through a filter to produce a fluorescent type quality with light appearing to emanate from the drawn line. The line drawing routine has a sort of ‘memory’ to it so the line drawn represents presence detected over more than one frame. The background is black which accentuates the neon-ness of the piece.

The sound design consists of matching percussion tracks featuring kick drum, snare and high hats. Moving in front of webcam/screen A activates the kick drum track, webcam/screen B the snare and C the hats. The idea being that if persistent movement is detected in front of each of the screens, the entire mix will play. Each drum channel is made up of a small number of phrases which trigger each other randomly so the mix as a whole can play for sometime without repetition.

Neon Dots (will be used if the  Carousela plug-in does not work)

Similar to above but instead of a neon line, dots are shown around the moving edge of a body. It looks quite nice if you haven’t seen it working with the Carousela plug-in.


Thick white lines are drawn around edges detected in the webcam image. Any movement results in a ghosting effect of the edges as frames are combined. Sparks fly from points of movement, the greater the movement the greater the number and velocity of the sparks. A gently cycling bass sequence provides sonic depth and some suspense while the visual sparks are also heard as 3 distinct collections of spark noise, linked to movement detected in respective webcam images. The three ‘spark wheels’ are panned left, centre and right to match the screens. I will need to tweak the sound design in situ to reinforce the relationship between sparks created by a given webcam and sounds emitted within the stereo field.


Frames of video are shown randomly from a ‘buffer’ of video frames taken over the preceding 24 frames or so. The effect is quite startling and looks like a video disc that keeps getting stuck, playing the same frame a few times before stuttering forward and backwards then repeating the process. Changes of luminance are tracked and when these are high enough (ie there is substantial movement) particles with trails are generated from the area of most movement. To me, these look a little like creases appearing on the image as would be seen on an old photograph and accentuate the sense of playing with time. Tracked movement is also linked to audio with increased movement increasing the frantic-ness of a glitchy staccato noise. Again, there are 3 noise generators, 1 for each screen, being linked directly to the movement detected in that particular screen. In contrast, an asynchronous sequence of chime-like sounds develops over the course of the piece. For me this is a way of counteracting the potential discomfort of otherwise glitchy visual combined with glitchy audio.

Rutt Ettra

This piece may yet be dropped as I am uncertain of how well the interaction works until able to calibrate in situ. Essentially the piece utilises a well known technique of mapping an image into 3D space using luminance to define extrusion in the Z plane (in this case provided by the Rutt Ettra plug-in). Audio input is picked up via the webcam microphone and used to radically increase the degree of Z extrusion. Audio is generated using an LFO to produce abstract phrases reminiscent of 1980s computer games. It may or may not work.


Ascii utilises the text mapping of a character set against luminance so that the brightness of an image fragment is matched to an ascii character – lighter areas being represented by characters at the beginning of the range (ie the alphabet), darker areas by characters towards the end. This in itself is not a new technique. In the vignette of the same name I have mixed a semi-opaque ascii luminance image map with an ‘edges’ image which helps to add definition of subject. A third uppermost layer shows larger characters corresponding to areas of image movement as they occur. The greater the range of movement the larger the character. The visual effect of moving is a little like creating a trail of characters. The sound design uses LFOs again to generate cascading phrases that modulate according to the level of detected movement. The timbre has a ‘computery’ feel and for me adds to the visual  perception of being surrounded by computer text or data. An asynchronous repetitive drone in the background might be the sound of a vast machine hidden from view but omnipresent.

Writing these vignette descriptions has helped me to think about what I might want to say about my work in the gallery and how I might say it.


Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s